-- Logs begin at Mon 2022-05-09 09:04:52 UTC, end at Mon 2022-05-09 10:43:19 UTC. -- May 09 09:32:35.607752 dv-cinder-841083-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 09 09:32:35.615373 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 09 09:32:35.622340 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 9 09:32:35 2022] *** May 09 09:32:35.622340 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 09 09:32:35.622340 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 09 09:32:35.622340 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: nodename: dv-cinder-841083-1-iscsi May 09 09:32:35.622340 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: machine: x86_64 May 09 09:32:35.622340 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: clock source: unix May 09 09:32:35.622340 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: pcre jit disabled May 09 09:32:35.622340 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: detected number of CPU cores: 4 May 09 09:32:35.622340 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: current working directory: / May 09 09:32:35.622340 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: detected binary path: /usr/bin/uwsgi-core May 09 09:32:35.622340 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: your processes number limit is 47680 May 09 09:32:35.622340 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: your memory page size is 4096 bytes May 09 09:32:35.622340 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: detected max file descriptor number: 2048 May 09 09:32:35.622340 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: lock engine: pthread robust mutexes May 09 09:32:35.622340 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: thunder lock: enabled May 09 09:32:35.622340 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 09 09:32:35.622340 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 09 09:32:35.663741 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: Python main interpreter initialized at 0x55768c1379c0 May 09 09:32:35.663741 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: python threads support enabled May 09 09:32:35.663741 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: your server socket listen backlog is limited to 100 connections May 09 09:32:35.663741 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: your mercy for graceful operations on workers is 90 seconds May 09 09:32:35.664111 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: mapped 403077 bytes (393 KB) for 2 cores May 09 09:32:35.664190 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: *** Operational MODE: preforking *** May 09 09:32:35.664450 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: *** uWSGI is running in multiple interpreter mode *** May 09 09:32:35.664450 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: spawned uWSGI master process (pid: 86584) May 09 09:32:35.664681 dv-cinder-841083-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 09 09:32:35.665693 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: spawned uWSGI worker 1 (pid: 86597, cores: 1) May 09 09:32:35.665693 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: spawned uWSGI worker 2 (pid: 86598, cores: 1) May 09 09:32:35.665693 dv-cinder-841083-1-iscsi devstack@keystone.service[86584]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 09 09:32:41.275020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 09:32:41.275020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: from cryptography.utils import int_from_bytes May 09 09:32:41.276155 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 09:32:41.276155 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: from cryptography.utils import int_from_bytes May 09 09:32:41.276638 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 09:32:41.276638 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: from cryptography.utils import int_from_bytes May 09 09:32:41.277545 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 09:32:41.277545 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: from cryptography.utils import int_from_bytes May 09 09:32:41.324098 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 09 09:32:41.324324 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 09 09:32:41.324535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] command line args: [] {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 09 09:32:41.324791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 09 09:32:41.324920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ================================================================================ {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 09 09:32:41.325088 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 09 09:32:41.325174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] admin_token = **** {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.325235 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 09 09:32:41.325464 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] config_dir = [] {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.325587 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] command line args: [] {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 09 09:32:41.325653 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.325823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] config_source = [] {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.325905 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 09 09:32:41.325972 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] debug = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.326032 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ================================================================================ {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 09 09:32:41.326184 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.328458 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] admin_token = **** {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.328530 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.328530 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.328641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] config_dir = [] {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.328739 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.328806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.328876 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.328876 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] list_limit = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.328991 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] config_source = [] {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.329050 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] log_config_append = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.329112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] debug = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.329187 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.329187 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] log_dir = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.329298 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.329377 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] log_file = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.329377 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.329489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.329547 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.329609 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.329667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.329667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.329775 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.329841 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.329841 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.329953 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.330010 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.330072 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] list_limit = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.330130 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.330197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] log_config_append = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.330255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.330311 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.330367 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.330428 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] log_dir = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.330485 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.330561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] log_file = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.330630 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.330693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.331297 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.332613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.332826 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.333064 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.333252 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.333446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.333446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.333446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.333960 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.334645 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.334877 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] publish_errors = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.335043 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.335273 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.335442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.335645 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.335832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.336059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.336220 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.336220 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.336590 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.336813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.336813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.338608 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.338792 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.338944 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] standard_threads = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.339082 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.339211 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.339339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] publish_errors = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.339339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.339661 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.339661 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.339909 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.340036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] use_journal = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.340179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.340331 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] use_json = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.340514 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.340738 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] use_stderr = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.340738 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] use_syslog = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.341019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.341156 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.341290 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] standard_threads = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.341290 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.341549 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.341683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.341812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.343704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.343848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.343980 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] use_journal = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.344114 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.344243 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] use_json = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.344243 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] use_stderr = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.344495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.344623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] use_syslog = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.344773 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.344903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 09:32:41.345052 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.345180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.345317 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.345457 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.345585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.345718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.345846 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.346057 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.347484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.348908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.349066 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.349202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.351567 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.351729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.351883 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.352018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.354412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.354563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.354563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.354838 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.354981 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.354981 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.355241 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.355375 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.355375 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.355738 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.355876 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.356024 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.356165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.356165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.356419 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.356562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.356714 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.356855 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.356855 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.357190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.357340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.357492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.357627 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.357767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.357918 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.358052 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.358192 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.358336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.358477 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.358622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.358755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.358889 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.359021 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.359166 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.359300 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.359433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.359566 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.359566 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.359830 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.359830 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.360164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.360297 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.360430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.360562 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.360745 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.360902 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.361037 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.361177 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.361310 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.361443 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.361585 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.361737 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.361872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.362011 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.362150 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.362283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.362424 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.362567 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.362708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.362849 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.362989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.363123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.363267 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.363401 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.363534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.365068 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.365282 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.365519 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.365723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.365897 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.366100 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.366295 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.366489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.366683 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.366886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.367091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.367286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.367479 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.367684 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.367878 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.368082 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.368279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.368540 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.368757 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.368934 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.369136 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.369364 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.369540 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.369540 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.369921 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.370118 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.370286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.370475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.370952 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.371092 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.371228 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.371363 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.371504 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.371677 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.371825 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.371962 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.372108 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.372254 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.372390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.372525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.372712 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.372856 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.373034 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.373179 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.373322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.373459 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.373593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.373737 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.373873 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.374014 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.374150 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.374292 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.374427 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.374580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.374716 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.374851 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.374986 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.375126 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.375262 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.375406 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.375543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.375713 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.375881 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.376025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.376162 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.376305 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.376442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.376586 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.376779 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.376918 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.377053 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.377328 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.377474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.377611 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.377746 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.377881 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.378016 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.378166 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.378303 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.378438 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.378584 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.378730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.378876 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.379011 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.379146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.379280 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.379414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.379550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.379693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.379829 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.379969 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.380104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.380238 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.380372 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.380524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.380709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.380867 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.381005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.381005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.381261 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.381395 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.381529 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.381674 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.381810 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.381951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.382087 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.382229 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.382365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.382522 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.382660 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.382795 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.382930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.382930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.383185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.383329 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.383465 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.383606 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.383743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.383743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] auth.external = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.383999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.384146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.384283 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.384283 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.384554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.384740 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] auth.external = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.384883 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.385054 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.385199 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] auth.password = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.385199 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] auth.token = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.385467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.385612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.385755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.385894 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.386035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] auth.password = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.386179 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.386325 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] auth.token = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.386461 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.386603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.386783 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.386932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.387069 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.387205 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.387340 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.387475 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.387610 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] credential.caching = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.387755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.387893 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.388028 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.388164 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.388310 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.388455 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.388599 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] credential.caching = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.388820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.388958 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.389094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.389228 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.389364 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.389510 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.389646 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.389781 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.389916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.390052 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.390197 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.390341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.390478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.390621 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.390757 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.390892 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.391027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.391181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.391318 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.391453 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.391588 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.391723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.391858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.391858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] federation.caching = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.392136 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.392272 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.392407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.392542 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.392699 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.392853 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.392990 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.393132 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.393270 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] federation.caching = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.393405 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.393541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.393751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.393889 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.394043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.394182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.394318 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.394459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.394603 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.394740 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.394876 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.395011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.395146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.395289 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.395434 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] identity.caching = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.395572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.395726 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.395985 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.396170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.396313 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.396457 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.396594 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.396762 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.396950 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] identity.caching = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.397092 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.397228 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.397376 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.397514 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.397650 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.397785 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.397920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.398065 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.398209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.398354 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.398492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.398627 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.398762 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.398905 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.399042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.399177 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.399313 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.399313 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.399571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.399715 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.399851 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.399851 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.400128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.400264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.400406 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.400556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.400760 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.400902 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.401037 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.401172 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.401307 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.401307 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.401577 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.401712 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.401860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.401996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.402132 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.402306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.402449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.402586 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.402722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.402857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.403001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.403145 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.403288 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.403424 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.403559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.403696 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.403696 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.403973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.404110 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.404245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.404387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.404524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.404677 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.404833 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.404971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.405107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.405243 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.405378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.405514 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.405657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.405808 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.405945 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.406080 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.406080 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.406336 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.406480 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.406624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.406759 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.406896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.407074 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.407214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.407214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.407481 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.407481 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.407748 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.407886 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.408022 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.408172 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.408311 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.408446 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.408592 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.408752 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.408891 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.409038 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.409175 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.409311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.409453 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.409599 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.user = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.409768 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.409917 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.410055 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.410196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.410333 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.410468 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.410604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.410748 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.410884 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.411021 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.411156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.411290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.411436 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.411580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.411717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.user = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.411852 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.411986 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.412122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.412257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.412407 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.412544 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.412732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.412876 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.413012 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.413160 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.413297 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.413443 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.413577 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.413713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.413847 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.413990 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.414127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.414263 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.414398 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.414533 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.414668 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.414811 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.414948 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.415083 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.415218 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.415363 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.415498 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.415641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.415777 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.415913 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.416047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.416182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.416318 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.416461 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.416597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.416754 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.416890 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.417026 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.417170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.417318 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.417455 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.417590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.417725 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.417945 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.418099 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.418247 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.418393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.418529 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.418664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.418799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.418935 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.419079 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.419227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.419364 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.419500 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.419642 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.419778 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.419921 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.420057 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] resource.caching = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.420196 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.420331 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.420505 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.420646 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.420896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.421036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.421171 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.421317 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.421460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.421596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.421740 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.421876 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.422011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.422255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.422423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] resource.caching = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.422563 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.422733 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.422916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.423166 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.423309 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] role.caching = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.423444 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.423591 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] role.driver = sql {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.423735 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.423871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.424006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.424140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.424275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.424411 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.424555 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.424712 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.424854 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.424989 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.425126 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.425270 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.425428 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] role.caching = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.425570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.425707 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] role.driver = sql {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.425842 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.425977 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.426111 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.426255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.426391 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.426391 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.426649 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.426786 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.426922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.427066 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.427213 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.427350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.427485 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.427485 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.427752 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.427889 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.428034 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.428177 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.428312 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.428448 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.428448 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.428759 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.428904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.429040 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.429187 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.429323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.429471 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.429608 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.429608 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.429867 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.430003 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.430138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.430282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.430418 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.430555 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.430690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.430825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.430825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.431107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.431243 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.431379 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.431514 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.431650 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.431786 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.431930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.432068 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.432203 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.432338 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.432473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.432608 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.432844 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.432984 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] token.caching = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.433120 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.433255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.433391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.433563 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.433717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.433854 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.433990 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.434126 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.434262 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.434411 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.434549 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.434684 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.434831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.434967 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.435104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.435249 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.435386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] token.caching = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.435521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.435656 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.435792 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.435927 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.436071 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.436209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.436345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.436480 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.436626 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.436782 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.436930 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.437067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.437204 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.437340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.437475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.437610 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.437754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.437900 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.438037 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] database.connection = **** {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.438182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.438328 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.438465 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.438465 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.438733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.438733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.439000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.439135 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.439272 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.439408 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.439544 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.439687 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.439824 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.439960 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.440095 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.440231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.440367 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] database.connection = **** {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.440525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.440690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.440834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.440970 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.441106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.441242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.441379 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.441526 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.441662 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.441798 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.441933 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.442068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.442203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.442346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.442494 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.442630 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.442765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.442901 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.443036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.443036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 09 09:32:41.443376 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.443376 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.443376 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.443376 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.443376 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.443376 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.443376 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.443376 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.443376 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 09:32:41.444501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86598) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 09 09:32:41.444638 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.444638 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.444638 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.445166 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.445166 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.445166 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.448655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.450401 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.476498 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.476738 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.478364 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.478596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.529104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.529368 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.580471 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.580724 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.583553 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.583763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.646863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.647165 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.647322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.647490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.647636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.647780 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.647924 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.648067 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.648218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.648218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.648503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.648503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.671193 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.671457 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.671618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.671767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.671910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.672053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.672197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.672349 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.672493 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.672648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.672844 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.672986 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.673143 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.673288 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.683851 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.684072 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.698634 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.698948 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.699113 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.699275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.699421 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.699564 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.699716 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.699869 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.700012 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.700155 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.700296 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.700438 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.700589 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86598) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.700769 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86597) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 09:32:41.717183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.717408 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 09:32:41.717563 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.717728 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.717874 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 09:32:41.718016 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 09:32:41.718195 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.718342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.718484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 09:32:41.718636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.718779 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.718919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.719075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.719215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.719370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.719370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.729864 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.730109 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.730275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.730424 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.730585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.730730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.730872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.731034 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.738825 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.739053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.742136 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.742350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.742507 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.742655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.742815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.742961 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.743104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.743260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.743405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.743548 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.743700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.743844 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.743985 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.744135 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.744282 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.744434 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.744578 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.744753 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.744911 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.745053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.745207 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.745350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.771751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.771974 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.773404 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.773616 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.773797 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.773957 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.779043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.779308 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.779475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.779635 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.779780 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.779924 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.786901 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.787113 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.787285 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.787437 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.791837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.792073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.792245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.792393 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.792547 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.792730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.792880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.793022 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.793164 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.793318 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.793461 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.793613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.793768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.793909 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.794060 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.794203 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.812235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.812457 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.812615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.812805 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.817020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.817461 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.820596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.820858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.821164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.821384 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.825858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.826398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.829424 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.829991 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.833146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.833435 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.833689 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.833873 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.834024 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.834258 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.834406 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.834550 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.834694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.834837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.834837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.835118 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.853391 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.853758 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.854007 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.854192 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.854344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.854489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.854633 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.854927 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.862177 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.862869 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.863904 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.864145 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.864597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.864874 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.867242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.867496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.867792 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.868000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.868158 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.868306 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.868463 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.868609 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.868811 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.869036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.882085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.882341 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.882635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.882843 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.883000 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.883161 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.883309 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.883453 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.883595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.883751 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.883905 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.884049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.884049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.884319 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.901668 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.902516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.904997 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.908264 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.908798 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.909908 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.911531 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.911805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.912087 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.912177 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.913814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.915480 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.915757 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.916050 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.919520 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.921126 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.924333 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.924614 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.924916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.925208 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.926116 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.928144 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.931404 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.931691 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.931967 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.932236 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.934404 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.934688 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.934957 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.941597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.941898 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.942171 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.942961 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86597) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.943242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.943517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.943789 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.944049 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.944320 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.944582 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.944884 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.945146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.945406 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.945668 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.945936 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.946193 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.946455 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86597) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.950502 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86598) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 09:32:41.950796 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.951074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.951344 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.951614 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.951903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.952170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.952441 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.952727 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.952994 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.953257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.953534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.953795 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.954051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86598) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 09:32:41.985720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55768c1379c0 pid: 86597 (default app) May 09 09:32:41.989240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1a90e-de30-42af-a089-9f0a4d0214c2 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:41.989535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1a90e-de30-42af-a089-9f0a4d0214c2 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:41.989820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1a90e-de30-42af-a089-9f0a4d0214c2 None None] PATH_INFO: `/v3` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:41.993412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55768c1379c0 pid: 86598 (default app) May 09 09:32:41.994043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Mon May 9 09:32:41 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:32:49.877210 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46bc2d-3451-4314-a5d9-06b2ab8081b0 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:49.877760 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46bc2d-3451-4314-a5d9-06b2ab8081b0 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:49.877760 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46bc2d-3451-4314-a5d9-06b2ab8081b0 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:49.882089 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:32:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:32:49.887717 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f0cb1-8146-4252-8b41-51f9abf36b22 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:49.888176 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f0cb1-8146-4252-8b41-51f9abf36b22 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:49.888255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f0cb1-8146-4252-8b41-51f9abf36b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:49.920205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG oslo_db.sqlalchemy.engines [None req-729f0cb1-8146-4252-8b41-51f9abf36b22 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86597) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 09 09:32:50.103337 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG passlib.handlers.bcrypt [None req-729f0cb1-8146-4252-8b41-51f9abf36b22 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86597) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 09 09:32:50.103757 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG passlib.handlers.bcrypt [None req-729f0cb1-8146-4252-8b41-51f9abf36b22 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86597) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 09 09:32:50.159964 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-729f0cb1-8146-4252-8b41-51f9abf36b22 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:32:50.161533 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-729f0cb1-8146-4252-8b41-51f9abf36b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:32:50.289906 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:32:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 09 09:32:50.469822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46bc2d-3451-4314-a5d9-06b2ab8081b0 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:32:50.505167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG oslo_db.sqlalchemy.engines [None req-3b46bc2d-3451-4314-a5d9-06b2ab8081b0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86598) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 09 09:32:50.629828 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46bc2d-3451-4314-a5d9-06b2ab8081b0 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:32:50.634519 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329e904-196f-4518-9c7e-1dac81c49ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:32:50.635174 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329e904-196f-4518-9c7e-1dac81c49ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:50.635451 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329e904-196f-4518-9c7e-1dac81c49ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:50.635718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329e904-196f-4518-9c7e-1dac81c49ac9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:50.643926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329e904-196f-4518-9c7e-1dac81c49ac9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:32:50.694350 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7329e904-196f-4518-9c7e-1dac81c49ac9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 09 09:32:50.694682 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329e904-196f-4518-9c7e-1dac81c49ac9 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:32:50.704238 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Mon May 9 09:32:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 09:32:50.720147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f0cb1-8146-4252-8b41-51f9abf36b22 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:32:50.734870 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f0cb1-8146-4252-8b41-51f9abf36b22 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:32:50.740788 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e911b6-91c2-4a49-af72-9a83153ffe0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:32:50.741970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e911b6-91c2-4a49-af72-9a83153ffe0f admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:50.742253 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e911b6-91c2-4a49-af72-9a83153ffe0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:50.742523 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e911b6-91c2-4a49-af72-9a83153ffe0f admin admin] PATH_INFO: `/v3/projects/03d8c548fb514d70b477284ceb7b3252` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:50.744188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e911b6-91c2-4a49-af72-9a83153ffe0f admin admin] RBAC: Authorizing `identity:get_project(project_id=03d8c548fb514d70b477284ceb7b3252)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:32:50.794224 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d7e911b6-91c2-4a49-af72-9a83153ffe0f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 09 09:32:50.794562 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e911b6-91c2-4a49-af72-9a83153ffe0f admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:32:50.799078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Mon May 9 09:32:50 2022] GET /identity/v3/projects/03d8c548fb514d70b477284ceb7b3252 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:32:52.974334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8341c6f-fb08-4332-840a-0d7bae4588e8 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:52.974876 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8341c6f-fb08-4332-840a-0d7bae4588e8 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:52.974876 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8341c6f-fb08-4332-840a-0d7bae4588e8 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:52.978934 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:32:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:32:52.985866 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f52d1e-273b-45d6-b5fc-b0942deba82c None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:52.986607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f52d1e-273b-45d6-b5fc-b0942deba82c None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:52.986756 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f52d1e-273b-45d6-b5fc-b0942deba82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:53.013162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-13f52d1e-273b-45d6-b5fc-b0942deba82c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:32:53.014749 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-13f52d1e-273b-45d6-b5fc-b0942deba82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:32:53.025062 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:32:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 09 09:32:53.047426 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8341c6f-fb08-4332-840a-0d7bae4588e8 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:32:53.060044 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8341c6f-fb08-4332-840a-0d7bae4588e8 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:32:53.064405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b319dad-224b-4f5a-abd4-d8bd037f1b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:32:53.065070 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b319dad-224b-4f5a-abd4-d8bd037f1b37 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:53.065349 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b319dad-224b-4f5a-abd4-d8bd037f1b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:53.065620 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b319dad-224b-4f5a-abd4-d8bd037f1b37 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:53.073366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b319dad-224b-4f5a-abd4-d8bd037f1b37 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:32:53.075165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b319dad-224b-4f5a-abd4-d8bd037f1b37 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:32:53.083916 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Mon May 9 09:32:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 09:32:53.089438 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f52d1e-273b-45d6-b5fc-b0942deba82c None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:32:53.098449 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f52d1e-273b-45d6-b5fc-b0942deba82c None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:32:53.104796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e6d3a-a752-42c8-90f5-7d8bf2ddb944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:32:53.105658 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e6d3a-a752-42c8-90f5-7d8bf2ddb944 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:53.105938 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e6d3a-a752-42c8-90f5-7d8bf2ddb944 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:53.106206 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e6d3a-a752-42c8-90f5-7d8bf2ddb944 admin admin] PATH_INFO: `/v3/users/68d9ab86ee454d7d9d1e4d3a48048d52` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:53.107811 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e6d3a-a752-42c8-90f5-7d8bf2ddb944 admin admin] RBAC: Authorizing `identity:get_user(user_id=68d9ab86ee454d7d9d1e4d3a48048d52)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:32:53.109691 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e6d3a-a752-42c8-90f5-7d8bf2ddb944 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:32:53.111243 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Mon May 9 09:32:53 2022] GET /identity/v3/users/68d9ab86ee454d7d9d1e4d3a48048d52 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:32:55.352946 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f76097-b7c9-4754-b705-19d7fc8e1716 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:55.353490 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f76097-b7c9-4754-b705-19d7fc8e1716 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:55.353490 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f76097-b7c9-4754-b705-19d7fc8e1716 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:55.355039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:32:55.360089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b2728-2fa2-4295-b169-a75748e6a83f None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:55.360529 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b2728-2fa2-4295-b169-a75748e6a83f None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:55.360805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b2728-2fa2-4295-b169-a75748e6a83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:55.404818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c842fe6-7cf2-41aa-899c-227e370408f3 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:55.405060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c842fe6-7cf2-41aa-899c-227e370408f3 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:55.405348 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c842fe6-7cf2-41aa-899c-227e370408f3 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:55.406687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 6/11] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:32:55.412746 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53992887-8e88-4149-955f-8c248a60b208 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:55.413026 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53992887-8e88-4149-955f-8c248a60b208 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:55.413294 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53992887-8e88-4149-955f-8c248a60b208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:55.454674 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG passlib.handlers.bcrypt [None req-53992887-8e88-4149-955f-8c248a60b208 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86598) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 09 09:32:55.455099 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG passlib.handlers.bcrypt [None req-53992887-8e88-4149-955f-8c248a60b208 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86598) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 09 09:32:55.488448 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-f25b2728-2fa2-4295-b169-a75748e6a83f None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:32:55.489697 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f25b2728-2fa2-4295-b169-a75748e6a83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:32:55.501987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-53992887-8e88-4149-955f-8c248a60b208 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:32:55.503575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-53992887-8e88-4149-955f-8c248a60b208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:32:55.570789 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 6/12] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:32:55.592827 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b2728-2fa2-4295-b169-a75748e6a83f None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:32:55.602140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 7/13] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:32:55.624761 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53992887-8e88-4149-955f-8c248a60b208 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:32:55.691956 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b2728-2fa2-4295-b169-a75748e6a83f None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:32:55.718351 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53992887-8e88-4149-955f-8c248a60b208 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:32:55.724523 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f8964-f44a-4036-928e-6299314d1f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:32:55.725343 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f8964-f44a-4036-928e-6299314d1f61 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:55.725621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f8964-f44a-4036-928e-6299314d1f61 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:55.725884 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f8964-f44a-4036-928e-6299314d1f61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:55.737277 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f8964-f44a-4036-928e-6299314d1f61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:32:55.748767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-da9f8964-f44a-4036-928e-6299314d1f61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:32:55.748767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:32:55.748767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:32:55.748767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:32:55.748767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:32:55.748767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:32:55.748767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:32:55.748767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:32:55.748767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:32:55.748767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:32:55.748767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:32:55.748767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:32:55.748767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:32:55.748767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:32:55.748767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:32:55.748767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:32:55.749920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:32:55.749920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:32:55.749920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:32:55.749920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:32:55.749920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:32:55.749920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:32:55.749920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:32:55.749920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:32:55.749920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:32:55.749920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:32:55.749920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:32:55.749920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:32:55.749920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:32:55.749920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:32:55.749920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:32:55.749920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:32:55.750971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:32:55.750971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:32:55.750971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:32:55.750971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:32:55.750971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:32:55.750971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:32:55.750971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:32:55.750971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:32:55.750971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:32:55.750971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:32:55.750971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:32:55.750971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:32:55.750971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 7/14] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:32:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:32:55.754684 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681551f4-b156-4d0c-becb-a1b18777e334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:32:55.755586 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681551f4-b156-4d0c-becb-a1b18777e334 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:55.755715 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f8964-f44a-4036-928e-6299314d1f61 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:32:55.755870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681551f4-b156-4d0c-becb-a1b18777e334 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:55.756139 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681551f4-b156-4d0c-becb-a1b18777e334 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:55.817442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681551f4-b156-4d0c-becb-a1b18777e334 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:32:55.818971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681551f4-b156-4d0c-becb-a1b18777e334 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:32:55.856069 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f8964-f44a-4036-928e-6299314d1f61 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:32:55.890053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6113924-650a-408c-8a3f-5d7c00106e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:32:55.890630 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6113924-650a-408c-8a3f-5d7c00106e07 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:55.890880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6113924-650a-408c-8a3f-5d7c00106e07 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:55.891171 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6113924-650a-408c-8a3f-5d7c00106e07 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:55.891910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6113924-650a-408c-8a3f-5d7c00106e07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:32:55.892207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6113924-650a-408c-8a3f-5d7c00106e07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:32:55.893616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6113924-650a-408c-8a3f-5d7c00106e07 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:32:55.905171 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 8/15] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:32:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:32:55.910704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6113924-650a-408c-8a3f-5d7c00106e07 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:32:55.917170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 8/16] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:32:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 09:32:55.922806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681551f4-b156-4d0c-becb-a1b18777e334 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:32:56.014458 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6113924-650a-408c-8a3f-5d7c00106e07 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:32:56.052675 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762abb65-6445-4a3f-8db9-cec01308f3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:32:56.053575 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762abb65-6445-4a3f-8db9-cec01308f3d7 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:56.053855 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762abb65-6445-4a3f-8db9-cec01308f3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:56.054124 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762abb65-6445-4a3f-8db9-cec01308f3d7 None admin] PATH_INFO: `/v3/users/68d9ab86ee454d7d9d1e4d3a48048d52` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:56.055426 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762abb65-6445-4a3f-8db9-cec01308f3d7 None admin] RBAC: Authorizing `identity:get_user(user_id=68d9ab86ee454d7d9d1e4d3a48048d52)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:32:56.056915 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762abb65-6445-4a3f-8db9-cec01308f3d7 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:32:56.058658 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 9/17] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:32:55 2022] GET /identity/v3/users/68d9ab86ee454d7d9d1e4d3a48048d52 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:32:56.064119 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762abb65-6445-4a3f-8db9-cec01308f3d7 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:32:56.125841 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681551f4-b156-4d0c-becb-a1b18777e334 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:32:56.158933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a897f9-d145-46eb-b0dd-6a4e858e1507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:32:56.159655 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a897f9-d145-46eb-b0dd-6a4e858e1507 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:56.159794 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762abb65-6445-4a3f-8db9-cec01308f3d7 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:32:56.159929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a897f9-d145-46eb-b0dd-6a4e858e1507 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:56.160173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a897f9-d145-46eb-b0dd-6a4e858e1507 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:56.178541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a897f9-d145-46eb-b0dd-6a4e858e1507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:32:56.180025 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a897f9-d145-46eb-b0dd-6a4e858e1507 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:32:56.192487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e5e47-bf6b-4ac2-903b-d2d5046294c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:32:56.193327 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e5e47-bf6b-4ac2-903b-d2d5046294c5 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:56.193607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e5e47-bf6b-4ac2-903b-d2d5046294c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:56.193900 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e5e47-bf6b-4ac2-903b-d2d5046294c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:56.195112 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e5e47-bf6b-4ac2-903b-d2d5046294c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:32:56.196577 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e5e47-bf6b-4ac2-903b-d2d5046294c5 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:32:56.198213 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 10/18] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:32:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:32:56.200199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-24a897f9-d145-46eb-b0dd-6a4e858e1507 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:32:56.200199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:32:56.200199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:32:56.200199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:32:56.200199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:32:56.200199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:32:56.200199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:32:56.200199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:32:56.200199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:32:56.200199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:32:56.200199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:32:56.200199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:32:56.200199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:32:56.200199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:32:56.200199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:32:56.201138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:32:56.201138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:32:56.201138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:32:56.201138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:32:56.201138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:32:56.201138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:32:56.201138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:32:56.201138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:32:56.201138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:32:56.201138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:32:56.201138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:32:56.201138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:32:56.201138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:32:56.201138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:32:56.201138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:32:56.201138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:32:56.201138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:32:56.201937 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:32:56.201937 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:32:56.201937 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:32:56.201937 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:32:56.201937 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 9/19] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:32:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:32:56.203903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e5e47-bf6b-4ac2-903b-d2d5046294c5 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:32:56.206961 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a897f9-d145-46eb-b0dd-6a4e858e1507 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:32:56.298293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a897f9-d145-46eb-b0dd-6a4e858e1507 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:32:56.302520 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e5e47-bf6b-4ac2-903b-d2d5046294c5 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:32:56.330542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f5b4a-d974-4b6d-bc5e-5e4459494a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:32:56.331174 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f5b4a-d974-4b6d-bc5e-5e4459494a78 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:56.331451 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f5b4a-d974-4b6d-bc5e-5e4459494a78 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:56.331754 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f5b4a-d974-4b6d-bc5e-5e4459494a78 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:56.332483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f5b4a-d974-4b6d-bc5e-5e4459494a78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:32:56.332805 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f5b4a-d974-4b6d-bc5e-5e4459494a78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:32:56.334216 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f5b4a-d974-4b6d-bc5e-5e4459494a78 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:32:56.335741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72f098-7891-4c12-b1d6-32e260a2331b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:32:56.336381 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72f098-7891-4c12-b1d6-32e260a2331b None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:56.336676 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72f098-7891-4c12-b1d6-32e260a2331b None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:56.336961 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72f098-7891-4c12-b1d6-32e260a2331b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:56.337774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72f098-7891-4c12-b1d6-32e260a2331b None admin] RBAC: Adding query filter params (role.id=70ac549382b14b2cbb8b35e5dfc648d4, scope.domain.id=default, user.id=68d9ab86ee454d7d9d1e4d3a48048d52) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:32:56.338075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72f098-7891-4c12-b1d6-32e260a2331b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:32:56.339476 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72f098-7891-4c12-b1d6-32e260a2331b None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:32:56.346090 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 11/20] 10.222.0.26 () {56 vars in 1322 bytes} [Mon May 9 09:32:56 2022] GET /identity/v3/role_assignments?user.id=68d9ab86ee454d7d9d1e4d3a48048d52&scope.domain.id=default&role.id=70ac549382b14b2cbb8b35e5dfc648d4 => generated 233 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:32:56.352206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 10/21] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:32:58.893290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28dc02-919a-409a-a5ad-fdf908485810 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:58.894278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28dc02-919a-409a-a5ad-fdf908485810 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:58.894777 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28dc02-919a-409a-a5ad-fdf908485810 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:58.896636 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 12/22] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:32:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:32:58.902287 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc8f38-5440-4b43-957b-a71dc97e6d4c None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:58.903142 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc8f38-5440-4b43-957b-a71dc97e6d4c None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:58.903633 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc8f38-5440-4b43-957b-a71dc97e6d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:58.934158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-06bc8f38-5440-4b43-957b-a71dc97e6d4c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:32:58.935415 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-06bc8f38-5440-4b43-957b-a71dc97e6d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:32:58.939159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9052d26-24bd-454e-a5cf-401cc21c4745 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:58.939444 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9052d26-24bd-454e-a5cf-401cc21c4745 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:58.939709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9052d26-24bd-454e-a5cf-401cc21c4745 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:58.941124 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 13/23] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:32:58.952036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17006bb9-7c5f-4f3c-ac0f-0fd24f6cd7a9 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:58.952320 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17006bb9-7c5f-4f3c-ac0f-0fd24f6cd7a9 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:58.952595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17006bb9-7c5f-4f3c-ac0f-0fd24f6cd7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:58.976913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-17006bb9-7c5f-4f3c-ac0f-0fd24f6cd7a9 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:32:58.978394 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-17006bb9-7c5f-4f3c-ac0f-0fd24f6cd7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:32:59.055110 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 11/24] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:32:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:32:59.087336 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc8f38-5440-4b43-957b-a71dc97e6d4c None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:32:59.087969 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 14/25] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:32:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:32:59.126940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17006bb9-7c5f-4f3c-ac0f-0fd24f6cd7a9 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:32:59.265604 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17006bb9-7c5f-4f3c-ac0f-0fd24f6cd7a9 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:32:59.300516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc8f38-5440-4b43-957b-a71dc97e6d4c None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:32:59.303604 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034da424-d7ff-483e-bc98-28ff35b40e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:32:59.304641 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034da424-d7ff-483e-bc98-28ff35b40e10 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:59.306094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034da424-d7ff-483e-bc98-28ff35b40e10 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:59.306574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034da424-d7ff-483e-bc98-28ff35b40e10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:59.330704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034da424-d7ff-483e-bc98-28ff35b40e10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:32:59.332722 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034da424-d7ff-483e-bc98-28ff35b40e10 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:32:59.369763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-034da424-d7ff-483e-bc98-28ff35b40e10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:32:59.369763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:32:59.369763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:32:59.369763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:32:59.369763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:32:59.369763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:32:59.369763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:32:59.369763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:32:59.369763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:32:59.369763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:32:59.369763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:32:59.369763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:32:59.369763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:32:59.369763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:32:59.369763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:32:59.370623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:32:59.370623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:32:59.370623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:32:59.370623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:32:59.370623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:32:59.370623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:32:59.370623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:32:59.370623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:32:59.370623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:32:59.370623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:32:59.370623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:32:59.370623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:32:59.370623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:32:59.370623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:32:59.370623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:32:59.370623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:32:59.370623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:32:59.371797 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:32:59.371797 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:32:59.371797 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:32:59.371797 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:32:59.380394 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 15/26] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:32:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:32:59.385661 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c150e0-8c0b-4209-b6ff-c6219687238a None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:59.385661 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c150e0-8c0b-4209-b6ff-c6219687238a None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:59.385661 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c150e0-8c0b-4209-b6ff-c6219687238a None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:59.385661 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 16/27] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:32:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:32:59.386782 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95e9da-82e2-4466-8aba-3c7336cb146c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:32:59.387740 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95e9da-82e2-4466-8aba-3c7336cb146c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:59.388149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95e9da-82e2-4466-8aba-3c7336cb146c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:59.388554 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95e9da-82e2-4466-8aba-3c7336cb146c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:59.390033 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c150e0-8c0b-4209-b6ff-c6219687238a None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:32:59.412104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95e9da-82e2-4466-8aba-3c7336cb146c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:32:59.423385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-ab95e9da-82e2-4466-8aba-3c7336cb146c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:32:59.423385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:32:59.423385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:32:59.423385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:32:59.423385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:32:59.423385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:32:59.423385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:32:59.423385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:32:59.423385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:32:59.423385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:32:59.423385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:32:59.423385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:32:59.423385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:32:59.423385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:32:59.423385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:32:59.423385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:32:59.424410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:32:59.424410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:32:59.424410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:32:59.424410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:32:59.424410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:32:59.424410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:32:59.424410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:32:59.424410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:32:59.424410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:32:59.424410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:32:59.424410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:32:59.424410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:32:59.424410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:32:59.424410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:32:59.424410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:32:59.424410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:32:59.425211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:32:59.425211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:32:59.425211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:32:59.425211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:32:59.425211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:32:59.425211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:32:59.425211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:32:59.425211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:32:59.425211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:32:59.425211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:32:59.425211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:32:59.425211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:32:59.437305 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 12/28] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:32:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:32:59.438969 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203aaff7-bba2-4f70-b86c-325852c06886 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:59.440565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203aaff7-bba2-4f70-b86c-325852c06886 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:59.441225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203aaff7-bba2-4f70-b86c-325852c06886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:59.478477 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-203aaff7-bba2-4f70-b86c-325852c06886 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:32:59.479902 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-203aaff7-bba2-4f70-b86c-325852c06886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:32:59.574136 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 13/29] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:32:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:32:59.574998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203aaff7-bba2-4f70-b86c-325852c06886 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:32:59.613880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c150e0-8c0b-4209-b6ff-c6219687238a None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:32:59.658418 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a219c-c17e-4bdc-85c9-5af0f133cac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:32:59.659625 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a219c-c17e-4bdc-85c9-5af0f133cac3 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:59.659967 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a219c-c17e-4bdc-85c9-5af0f133cac3 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:59.660390 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a219c-c17e-4bdc-85c9-5af0f133cac3 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:59.661335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a219c-c17e-4bdc-85c9-5af0f133cac3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:32:59.661640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a219c-c17e-4bdc-85c9-5af0f133cac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:32:59.664097 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a219c-c17e-4bdc-85c9-5af0f133cac3 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:32:59.687617 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 17/30] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:32:59.688455 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a219c-c17e-4bdc-85c9-5af0f133cac3 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:32:59.696297 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203aaff7-bba2-4f70-b86c-325852c06886 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:32:59.738135 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b0a18-34b9-47fa-adf0-2c708fc1653e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:32:59.739106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b0a18-34b9-47fa-adf0-2c708fc1653e None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:59.739638 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b0a18-34b9-47fa-adf0-2c708fc1653e None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:59.739917 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b0a18-34b9-47fa-adf0-2c708fc1653e None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:59.740690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b0a18-34b9-47fa-adf0-2c708fc1653e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:32:59.740999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b0a18-34b9-47fa-adf0-2c708fc1653e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:32:59.743273 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b0a18-34b9-47fa-adf0-2c708fc1653e None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:32:59.758915 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 14/31] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:32:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:32:59.760400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc696ed-007d-47ad-bc2e-f086ea65770b None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:59.760717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc696ed-007d-47ad-bc2e-f086ea65770b None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:59.760995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc696ed-007d-47ad-bc2e-f086ea65770b None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:59.764742 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 15/32] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:32:59.765601 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc696ed-007d-47ad-bc2e-f086ea65770b None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:32:59.815815 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a219c-c17e-4bdc-85c9-5af0f133cac3 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:32:59.849006 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a480193-25f5-4546-aa67-44c402895d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:32:59.849718 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a480193-25f5-4546-aa67-44c402895d17 None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:59.849991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a480193-25f5-4546-aa67-44c402895d17 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:59.850258 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a480193-25f5-4546-aa67-44c402895d17 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:59.851096 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a480193-25f5-4546-aa67-44c402895d17 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:32:59.852520 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a480193-25f5-4546-aa67-44c402895d17 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:32:59.889629 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc696ed-007d-47ad-bc2e-f086ea65770b None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:32:59.923999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716eaa9-c4db-49f4-8e35-aa89ff530902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:32:59.924794 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716eaa9-c4db-49f4-8e35-aa89ff530902 None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:32:59.924992 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716eaa9-c4db-49f4-8e35-aa89ff530902 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:32:59.925272 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716eaa9-c4db-49f4-8e35-aa89ff530902 None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:32:59.926259 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716eaa9-c4db-49f4-8e35-aa89ff530902 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:32:59.927775 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716eaa9-c4db-49f4-8e35-aa89ff530902 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:32:59.985152 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 18/33] 10.222.0.26 () {60 vars in 1131 bytes} [Mon May 9 09:32:59 2022] POST /identity/v3/roles => generated 219 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:32:59.986346 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a480193-25f5-4546-aa67-44c402895d17 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:00.073394 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 16/34] 10.222.0.26 () {60 vars in 1141 bytes} [Mon May 9 09:32:59 2022] POST /identity/v3/projects => generated 301 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:33:00.075159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbba611-f4fe-4a7e-8264-f6624a1f5a2a None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:00.075451 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbba611-f4fe-4a7e-8264-f6624a1f5a2a None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:00.075729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbba611-f4fe-4a7e-8264-f6624a1f5a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:00.112711 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-4fbba611-f4fe-4a7e-8264-f6624a1f5a2a None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:00.114569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-4fbba611-f4fe-4a7e-8264-f6624a1f5a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:00.199169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a480193-25f5-4546-aa67-44c402895d17 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:00.220369 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 17/35] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:00.221796 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ee573-bef6-43fa-b016-11c924d28e35 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:00.222086 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ee573-bef6-43fa-b016-11c924d28e35 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:00.222362 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ee573-bef6-43fa-b016-11c924d28e35 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:00.223805 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 18/36] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:00.229355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988994b-d39a-40b6-9680-48de41ee946e None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:00.229640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988994b-d39a-40b6-9680-48de41ee946e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:00.229917 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988994b-d39a-40b6-9680-48de41ee946e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:00.260404 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-6988994b-d39a-40b6-9680-48de41ee946e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:00.263619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-6988994b-d39a-40b6-9680-48de41ee946e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:00.283166 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725d569-4e8a-4e4c-b534-70510ba78680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:00.284180 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725d569-4e8a-4e4c-b534-70510ba78680 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:00.284466 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725d569-4e8a-4e4c-b534-70510ba78680 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:00.284767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725d569-4e8a-4e4c-b534-70510ba78680 None admin] PATH_INFO: `/v3/users/68d9ab86ee454d7d9d1e4d3a48048d52` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:00.286578 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725d569-4e8a-4e4c-b534-70510ba78680 None admin] RBAC: Authorizing `identity:get_user(user_id=68d9ab86ee454d7d9d1e4d3a48048d52)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:00.289852 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725d569-4e8a-4e4c-b534-70510ba78680 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:00.294493 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 19/37] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:32:59 2022] GET /identity/v3/users/68d9ab86ee454d7d9d1e4d3a48048d52 => generated 247 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:00.295407 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725d569-4e8a-4e4c-b534-70510ba78680 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:00.382870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 19/38] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:00.382870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988994b-d39a-40b6-9680-48de41ee946e None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:00.444179 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725d569-4e8a-4e4c-b534-70510ba78680 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:00.489999 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272d7e2-6da8-424a-99b7-c57774b8b201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:00.490671 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272d7e2-6da8-424a-99b7-c57774b8b201 None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:00.490949 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272d7e2-6da8-424a-99b7-c57774b8b201 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:00.491218 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272d7e2-6da8-424a-99b7-c57774b8b201 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:00.491972 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8272d7e2-6da8-424a-99b7-c57774b8b201 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:00.493451 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8272d7e2-6da8-424a-99b7-c57774b8b201 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:00.506855 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988994b-d39a-40b6-9680-48de41ee946e None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:00.508199 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 20/39] 10.222.0.26 () {60 vars in 1131 bytes} [Mon May 9 09:33:00 2022] POST /identity/v3/roles => generated 225 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:33:00.509017 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272d7e2-6da8-424a-99b7-c57774b8b201 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:00.562850 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cfb74-8a57-4a5f-9b00-eb73f3ba8263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:00.564482 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cfb74-8a57-4a5f-9b00-eb73f3ba8263 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:00.564978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cfb74-8a57-4a5f-9b00-eb73f3ba8263 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:00.565427 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cfb74-8a57-4a5f-9b00-eb73f3ba8263 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:00.567598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304cfb74-8a57-4a5f-9b00-eb73f3ba8263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:00.570074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304cfb74-8a57-4a5f-9b00-eb73f3ba8263 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:00.572920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 20/40] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:33:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:00.578888 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cfb74-8a57-4a5f-9b00-eb73f3ba8263 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:00.649473 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272d7e2-6da8-424a-99b7-c57774b8b201 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:00.690309 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733932d6-9810-45b7-be72-7ecb6b9d100e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:00.691600 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733932d6-9810-45b7-be72-7ecb6b9d100e None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:00.692083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733932d6-9810-45b7-be72-7ecb6b9d100e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:00.692933 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733932d6-9810-45b7-be72-7ecb6b9d100e None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:00.694155 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733932d6-9810-45b7-be72-7ecb6b9d100e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:00.696285 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733932d6-9810-45b7-be72-7ecb6b9d100e None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:00.713505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 21/41] 10.222.0.26 () {60 vars in 1131 bytes} [Mon May 9 09:33:00 2022] POST /identity/v3/roles => generated 223 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:33:00.713932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cfb74-8a57-4a5f-9b00-eb73f3ba8263 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:00.757352 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85328bb-9514-45eb-a93d-fe27ae8b487e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:00.758371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85328bb-9514-45eb-a93d-fe27ae8b487e None admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:00.758645 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85328bb-9514-45eb-a93d-fe27ae8b487e None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:00.758971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85328bb-9514-45eb-a93d-fe27ae8b487e None admin] PATH_INFO: `/v3/domains/default/users/68d9ab86ee454d7d9d1e4d3a48048d52/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:00.761572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85328bb-9514-45eb-a93d-fe27ae8b487e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=68d9ab86ee454d7d9d1e4d3a48048d52, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:00.763176 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85328bb-9514-45eb-a93d-fe27ae8b487e None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:00.781820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 21/42] 10.222.0.26 () {58 vars in 1361 bytes} [Mon May 9 09:33:00 2022] PUT /identity/v3/domains/default/users/68d9ab86ee454d7d9d1e4d3a48048d52/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:33:00.830942 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e49cd9-a247-45e3-a3b2-8ef2f4579bcd None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:00.831296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e49cd9-a247-45e3-a3b2-8ef2f4579bcd None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:00.831510 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e49cd9-a247-45e3-a3b2-8ef2f4579bcd None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:00.833136 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 22/43] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:00.838895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d304e8e-208d-4f51-af76-8e8f02bef99d None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:00.839225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d304e8e-208d-4f51-af76-8e8f02bef99d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:00.839467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d304e8e-208d-4f51-af76-8e8f02bef99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:00.858085 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-3d304e8e-208d-4f51-af76-8e8f02bef99d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:00.859345 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-3d304e8e-208d-4f51-af76-8e8f02bef99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:00.931872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 22/44] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:00.955429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e49cd9-a247-45e3-a3b2-8ef2f4579bcd None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:01.055437 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e49cd9-a247-45e3-a3b2-8ef2f4579bcd None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:01.088546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af47bb-bd0b-4a14-a1b0-329ca3b1ce76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:01.089365 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af47bb-bd0b-4a14-a1b0-329ca3b1ce76 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:01.089686 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af47bb-bd0b-4a14-a1b0-329ca3b1ce76 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:01.089959 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af47bb-bd0b-4a14-a1b0-329ca3b1ce76 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:01.091257 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af47bb-bd0b-4a14-a1b0-329ca3b1ce76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:01.092692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af47bb-bd0b-4a14-a1b0-329ca3b1ce76 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:01.094330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 23/45] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:33:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:01.100046 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d304e8e-208d-4f51-af76-8e8f02bef99d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:01.195795 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d304e8e-208d-4f51-af76-8e8f02bef99d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:01.228089 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d3db4-2405-4fb2-9e3a-e8d1e8a0a14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:01.228775 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d3db4-2405-4fb2-9e3a-e8d1e8a0a14a None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:01.229062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d3db4-2405-4fb2-9e3a-e8d1e8a0a14a None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:01.229325 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d3db4-2405-4fb2-9e3a-e8d1e8a0a14a None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:01.230030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d3db4-2405-4fb2-9e3a-e8d1e8a0a14a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:01.231523 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d3db4-2405-4fb2-9e3a-e8d1e8a0a14a None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:01.259689 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 23/46] 10.222.0.26 () {60 vars in 1141 bytes} [Mon May 9 09:33:01 2022] POST /identity/v3/projects => generated 312 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:33:03.006131 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e050bdb-5bd3-4788-82c2-0c6b414610b4 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:03.006648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e050bdb-5bd3-4788-82c2-0c6b414610b4 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:03.006648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e050bdb-5bd3-4788-82c2-0c6b414610b4 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:03.007945 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 24/47] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:03.013529 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed2be3-f9dd-4c2e-b0c3-6a96f36e436c None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:03.013934 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed2be3-f9dd-4c2e-b0c3-6a96f36e436c None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:03.014119 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed2be3-f9dd-4c2e-b0c3-6a96f36e436c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:03.031202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-5aed2be3-f9dd-4c2e-b0c3-6a96f36e436c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:03.032375 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-5aed2be3-f9dd-4c2e-b0c3-6a96f36e436c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:03.097508 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 24/48] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:03.120374 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e050bdb-5bd3-4788-82c2-0c6b414610b4 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:03.230387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e050bdb-5bd3-4788-82c2-0c6b414610b4 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:03.263926 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cfe50-b31c-4ac1-9d52-ff9aa794ba8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:03.264652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cfe50-b31c-4ac1-9d52-ff9aa794ba8e None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:03.264963 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cfe50-b31c-4ac1-9d52-ff9aa794ba8e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:03.265233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cfe50-b31c-4ac1-9d52-ff9aa794ba8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:03.277667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987cfe50-b31c-4ac1-9d52-ff9aa794ba8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:03.285538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-987cfe50-b31c-4ac1-9d52-ff9aa794ba8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:33:03.285538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:03.285538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:03.285538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:03.285538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:03.285538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:03.285538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:03.285538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:03.285538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:03.285538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:03.285538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:03.285538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:03.285538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:03.285538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:03.285538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:03.285538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:03.286570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:03.286570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:03.286570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:03.286570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:03.286570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:03.286570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:03.286570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:03.286570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:03.286570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:03.286570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:03.286570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:03.286570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:33:03.286570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:03.286570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:03.286570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:03.286570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:03.287339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:03.287339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:03.287339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:03.287339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:03.287339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:03.287339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:03.287339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:03.287339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:03.287339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:03.287339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:03.287339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:33:03.287339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:33:03.287339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 25/49] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:33:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:03.293339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed2be3-f9dd-4c2e-b0c3-6a96f36e436c None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:03.389632 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed2be3-f9dd-4c2e-b0c3-6a96f36e436c None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:03.423191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06b76a-02b4-4c5d-88f2-9f94dd2c3afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:03.423857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06b76a-02b4-4c5d-88f2-9f94dd2c3afa None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:03.424140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06b76a-02b4-4c5d-88f2-9f94dd2c3afa None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:03.424411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06b76a-02b4-4c5d-88f2-9f94dd2c3afa None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:03.425211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06b76a-02b4-4c5d-88f2-9f94dd2c3afa None admin] RBAC: Adding query filter params (name=admin) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:03.425519 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06b76a-02b4-4c5d-88f2-9f94dd2c3afa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:03.426891 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06b76a-02b4-4c5d-88f2-9f94dd2c3afa None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:03.435003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 25/50] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:03.440856 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cfe50-b31c-4ac1-9d52-ff9aa794ba8e None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:03.454067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7f645-3fa2-49c1-8d20-ff512c7c9ab0 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:03.454354 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7f645-3fa2-49c1-8d20-ff512c7c9ab0 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:03.454628 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7f645-3fa2-49c1-8d20-ff512c7c9ab0 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:03.456006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 26/51] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:03.460926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9d07d-a7bb-4a73-86c2-34f54f6dcc8d None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:03.461212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9d07d-a7bb-4a73-86c2-34f54f6dcc8d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:03.461483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9d07d-a7bb-4a73-86c2-34f54f6dcc8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:03.481005 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-1ad9d07d-a7bb-4a73-86c2-34f54f6dcc8d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:03.482197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-1ad9d07d-a7bb-4a73-86c2-34f54f6dcc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:03.539802 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cfe50-b31c-4ac1-9d52-ff9aa794ba8e None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:03.551029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 27/52] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:03.571564 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12ee70-bedf-42d2-83c3-2ee2c9aa046d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:03.572705 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12ee70-bedf-42d2-83c3-2ee2c9aa046d None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:03.573034 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12ee70-bedf-42d2-83c3-2ee2c9aa046d None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:03.573312 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12ee70-bedf-42d2-83c3-2ee2c9aa046d None admin] PATH_INFO: `/v3/users/68d9ab86ee454d7d9d1e4d3a48048d52` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:03.573459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9d07d-a7bb-4a73-86c2-34f54f6dcc8d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:03.574576 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c12ee70-bedf-42d2-83c3-2ee2c9aa046d None admin] RBAC: Authorizing `identity:get_user(user_id=68d9ab86ee454d7d9d1e4d3a48048d52)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:03.576038 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c12ee70-bedf-42d2-83c3-2ee2c9aa046d None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:03.577682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 26/53] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:03 2022] GET /identity/v3/users/68d9ab86ee454d7d9d1e4d3a48048d52 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:03.582944 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12ee70-bedf-42d2-83c3-2ee2c9aa046d None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:03.670257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9d07d-a7bb-4a73-86c2-34f54f6dcc8d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:03.677906 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12ee70-bedf-42d2-83c3-2ee2c9aa046d None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:03.703569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde94871-26a6-48fc-93c8-5e7d15c7c8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:03.704372 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde94871-26a6-48fc-93c8-5e7d15c7c8c4 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:03.704652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde94871-26a6-48fc-93c8-5e7d15c7c8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:03.704955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde94871-26a6-48fc-93c8-5e7d15c7c8c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:03.706154 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde94871-26a6-48fc-93c8-5e7d15c7c8c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:03.707619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde94871-26a6-48fc-93c8-5e7d15c7c8c4 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:03.709277 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 28/54] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:33:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:03.710557 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25819d9f-2c3b-4a39-8da6-a87a21edb096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:03.711342 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25819d9f-2c3b-4a39-8da6-a87a21edb096 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:03.711622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25819d9f-2c3b-4a39-8da6-a87a21edb096 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:03.711893 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25819d9f-2c3b-4a39-8da6-a87a21edb096 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:03.713100 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25819d9f-2c3b-4a39-8da6-a87a21edb096 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:03.714548 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25819d9f-2c3b-4a39-8da6-a87a21edb096 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:03.714910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde94871-26a6-48fc-93c8-5e7d15c7c8c4 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:03.716177 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 27/55] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:33:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:03.721595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25819d9f-2c3b-4a39-8da6-a87a21edb096 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:03.808826 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde94871-26a6-48fc-93c8-5e7d15c7c8c4 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:03.815776 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25819d9f-2c3b-4a39-8da6-a87a21edb096 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:03.842507 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c83d23-18d5-480a-952c-9baa5b4f673d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:03.843191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c83d23-18d5-480a-952c-9baa5b4f673d None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:03.843453 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c83d23-18d5-480a-952c-9baa5b4f673d None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:03.843715 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c83d23-18d5-480a-952c-9baa5b4f673d None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:03.844455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c83d23-18d5-480a-952c-9baa5b4f673d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:03.845955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c83d23-18d5-480a-952c-9baa5b4f673d None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:03.847734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f6c7d-fdac-4423-b6a3-e55fda18665a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:03.848374 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f6c7d-fdac-4423-b6a3-e55fda18665a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:03.848646 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f6c7d-fdac-4423-b6a3-e55fda18665a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:03.848956 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f6c7d-fdac-4423-b6a3-e55fda18665a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:03.849775 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f6c7d-fdac-4423-b6a3-e55fda18665a None admin] RBAC: Adding query filter params (role.id=70ac549382b14b2cbb8b35e5dfc648d4, scope.domain.id=default, user.id=68d9ab86ee454d7d9d1e4d3a48048d52) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:03.850084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f6c7d-fdac-4423-b6a3-e55fda18665a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:03.851493 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f6c7d-fdac-4423-b6a3-e55fda18665a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:03.857382 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 28/56] 10.222.0.26 () {56 vars in 1322 bytes} [Mon May 9 09:33:03 2022] GET /identity/v3/role_assignments?user.id=68d9ab86ee454d7d9d1e4d3a48048d52&scope.domain.id=default&role.id=70ac549382b14b2cbb8b35e5dfc648d4 => generated 530 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:03.871869 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 29/57] 10.222.0.26 () {60 vars in 1140 bytes} [Mon May 9 09:33:03 2022] POST /identity/v3/projects => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:33:06.072982 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bc1f1-c473-4ef3-a73a-449a2c91546e None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:06.073570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bc1f1-c473-4ef3-a73a-449a2c91546e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:06.073570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bc1f1-c473-4ef3-a73a-449a2c91546e None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:06.074949 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 29/58] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:06.080391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9fb16-75e8-415d-bf0c-40bc54db3c89 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:06.080952 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9fb16-75e8-415d-bf0c-40bc54db3c89 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:06.080952 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9fb16-75e8-415d-bf0c-40bc54db3c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:06.098628 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-4ce9fb16-75e8-415d-bf0c-40bc54db3c89 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:06.100129 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-4ce9fb16-75e8-415d-bf0c-40bc54db3c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:06.166280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 30/59] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:06.190544 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bc1f1-c473-4ef3-a73a-449a2c91546e None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:06.296081 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bc1f1-c473-4ef3-a73a-449a2c91546e None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:06.329655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98944144-d5c6-45d7-91b5-47a17c59f6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:06.330360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98944144-d5c6-45d7-91b5-47a17c59f6df None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:06.330636 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98944144-d5c6-45d7-91b5-47a17c59f6df None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:06.330905 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98944144-d5c6-45d7-91b5-47a17c59f6df None admin] PATH_INFO: `/v3/domains/default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:06.332186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98944144-d5c6-45d7-91b5-47a17c59f6df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:06.333654 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98944144-d5c6-45d7-91b5-47a17c59f6df None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:06.335227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 30/60] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:33:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:06.340574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9fb16-75e8-415d-bf0c-40bc54db3c89 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:06.441984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9fb16-75e8-415d-bf0c-40bc54db3c89 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:06.477426 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314632d2-2a26-4b84-8c4a-c557773d0127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:06.478088 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314632d2-2a26-4b84-8c4a-c557773d0127 None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:06.478372 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314632d2-2a26-4b84-8c4a-c557773d0127 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:06.478646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314632d2-2a26-4b84-8c4a-c557773d0127 None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:06.479365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314632d2-2a26-4b84-8c4a-c557773d0127 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:06.480856 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314632d2-2a26-4b84-8c4a-c557773d0127 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:06.512923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 31/61] 10.222.0.26 () {60 vars in 1132 bytes} [Mon May 9 09:33:06 2022] POST /identity/v3/users => generated 275 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:33:09.824803 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e245df-6367-437b-a045-e2c8ac60fc4a None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:09.824803 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e245df-6367-437b-a045-e2c8ac60fc4a None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:09.824803 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e245df-6367-437b-a045-e2c8ac60fc4a None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:09.824803 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 31/62] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:09.829962 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c279a-eeae-4ce6-84b8-c1db75edde43 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:09.830359 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c279a-eeae-4ce6-84b8-c1db75edde43 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:09.830506 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c279a-eeae-4ce6-84b8-c1db75edde43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:09.853675 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-e52c279a-eeae-4ce6-84b8-c1db75edde43 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:09.855223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e52c279a-eeae-4ce6-84b8-c1db75edde43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:09.938566 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 32/63] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:09.960487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6035ea-aa01-4297-a85f-4eaea6fca28a None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:09.961058 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6035ea-aa01-4297-a85f-4eaea6fca28a None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:09.961492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6035ea-aa01-4297-a85f-4eaea6fca28a None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:09.963502 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 32/64] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:09.969546 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1708a5f-0163-48a6-be51-3ce9085cfdbf None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:09.970063 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1708a5f-0163-48a6-be51-3ce9085cfdbf None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:09.970765 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1708a5f-0163-48a6-be51-3ce9085cfdbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:09.971520 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6035ea-aa01-4297-a85f-4eaea6fca28a None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:10.010181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-e1708a5f-0163-48a6-be51-3ce9085cfdbf None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:10.011658 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e1708a5f-0163-48a6-be51-3ce9085cfdbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:10.102642 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6035ea-aa01-4297-a85f-4eaea6fca28a None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:10.139543 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c384a78-4c8a-4803-b4a0-6e48d620463f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:10.140345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c384a78-4c8a-4803-b4a0-6e48d620463f None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:10.140623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c384a78-4c8a-4803-b4a0-6e48d620463f None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:10.140922 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c384a78-4c8a-4803-b4a0-6e48d620463f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:10.151745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c384a78-4c8a-4803-b4a0-6e48d620463f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:10.160043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-4c384a78-4c8a-4803-b4a0-6e48d620463f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:33:10.160043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:10.160043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:10.160043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:10.160043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:10.160043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:10.160043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:10.160043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:10.160043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:10.160043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:10.160043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:10.160043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:10.160043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:10.160043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:10.160043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:10.160043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:10.160951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:10.160951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:10.160951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:10.160951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:10.160951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:10.160951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:10.160951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:10.160951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:10.160951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:10.160951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:10.160951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:10.160951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:33:10.160951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:10.160951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:10.160951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:10.160951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:10.161796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:10.161796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:10.161796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:10.161796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:10.161796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:10.161796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:10.161796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:10.161796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:10.161796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:10.161796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:10.161796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:33:10.161796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:33:10.161796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 33/65] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:33:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:33:10.168128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c384a78-4c8a-4803-b4a0-6e48d620463f None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:10.180801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 33/66] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:10.210426 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40584b22-9584-496c-9bc3-978859f29159 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:10.210737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40584b22-9584-496c-9bc3-978859f29159 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:10.210991 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40584b22-9584-496c-9bc3-978859f29159 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:10.212677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 34/67] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:10.218434 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8450db-1bc7-4b1f-983b-6289e46e907e None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:10.218720 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8450db-1bc7-4b1f-983b-6289e46e907e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:10.218978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8450db-1bc7-4b1f-983b-6289e46e907e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:10.239793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-9f8450db-1bc7-4b1f-983b-6289e46e907e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:10.241313 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-9f8450db-1bc7-4b1f-983b-6289e46e907e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:10.305121 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c384a78-4c8a-4803-b4a0-6e48d620463f None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:10.325068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 35/68] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 09 09:33:10.325966 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8450db-1bc7-4b1f-983b-6289e46e907e None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:10.357882 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c26a1b-d0bd-4c94-a8e1-6998743dde03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:10.358504 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c26a1b-d0bd-4c94-a8e1-6998743dde03 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:10.358766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c26a1b-d0bd-4c94-a8e1-6998743dde03 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:10.359038 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c26a1b-d0bd-4c94-a8e1-6998743dde03 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:10.360085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c26a1b-d0bd-4c94-a8e1-6998743dde03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:10.360391 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c26a1b-d0bd-4c94-a8e1-6998743dde03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:10.361835 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c26a1b-d0bd-4c94-a8e1-6998743dde03 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:10.373685 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 34/69] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:10.374328 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c26a1b-d0bd-4c94-a8e1-6998743dde03 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:10.439659 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8450db-1bc7-4b1f-983b-6289e46e907e None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:10.471283 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9924a12-2bb0-4ba4-ae98-0de4020a0b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:10.472086 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9924a12-2bb0-4ba4-ae98-0de4020a0b39 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:10.472368 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9924a12-2bb0-4ba4-ae98-0de4020a0b39 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:10.472644 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9924a12-2bb0-4ba4-ae98-0de4020a0b39 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:10.478056 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c26a1b-d0bd-4c94-a8e1-6998743dde03 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:10.482460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9924a12-2bb0-4ba4-ae98-0de4020a0b39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:10.490383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-e9924a12-2bb0-4ba4-ae98-0de4020a0b39 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:10.490383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:10.490383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:10.490383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:10.490383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:10.490383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:10.490383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:10.490383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:10.490383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:10.490383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:10.490383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:10.490383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:10.490383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:10.490383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:10.490383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:10.490383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:10.491425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:10.491425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:10.491425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:10.491425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:10.491425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:10.491425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:10.491425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:10.491425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:10.491425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:10.491425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:10.491425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:10.491425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:10.491425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:10.491425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:10.491425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:10.491425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:10.492378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:10.492378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:10.492378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:10.492378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:10.492378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:10.492378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:10.492378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:10.492378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:10.492378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:10.492378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:10.492378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:10.492378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:10.499015 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 36/70] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:10.499015 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9924a12-2bb0-4ba4-ae98-0de4020a0b39 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:10.514471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7bcd4-6901-4d9f-a04a-dc9647e1fad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:10.515273 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7bcd4-6901-4d9f-a04a-dc9647e1fad4 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:10.515550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7bcd4-6901-4d9f-a04a-dc9647e1fad4 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:10.515816 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7bcd4-6901-4d9f-a04a-dc9647e1fad4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:10.526565 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7bcd4-6901-4d9f-a04a-dc9647e1fad4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:10.534478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-20b7bcd4-6901-4d9f-a04a-dc9647e1fad4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:10.534478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:10.534478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:10.534478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:10.534478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:10.534478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:10.534478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:10.534478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:10.534478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:10.534478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:10.534478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:10.534478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:10.534478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:10.534478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:10.534478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:10.534478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:10.535330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:10.535330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:10.535330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:10.535330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:10.535330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:10.535330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:10.535330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:10.535330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:10.535330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:10.535330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:10.535330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:10.535330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:33:10.535330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:10.535330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:10.535330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:10.535330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:10.536087 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:10.536087 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:10.536087 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:10.536087 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:10.536087 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:10.536087 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:10.536087 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:10.536087 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:10.536087 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:10.536087 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:10.536087 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:10.536087 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:33:10.536087 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 35/71] 10.222.0.26 () {56 vars in 1114 bytes} [Mon May 9 09:33:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:10.537245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692df6c2-6e5e-4bd8-b18a-7972c3323121 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:10.537245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692df6c2-6e5e-4bd8-b18a-7972c3323121 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:10.537402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692df6c2-6e5e-4bd8-b18a-7972c3323121 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:10.538609 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 36/72] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:10.539286 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692df6c2-6e5e-4bd8-b18a-7972c3323121 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:10.600860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9924a12-2bb0-4ba4-ae98-0de4020a0b39 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:10.633030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3daa4f2-976f-4f08-926b-585d223e7db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:10.633826 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3daa4f2-976f-4f08-926b-585d223e7db5 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:10.634103 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3daa4f2-976f-4f08-926b-585d223e7db5 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:10.634371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3daa4f2-976f-4f08-926b-585d223e7db5 None admin] PATH_INFO: `/v3/users/68d9ab86ee454d7d9d1e4d3a48048d52` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:10.635577 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3daa4f2-976f-4f08-926b-585d223e7db5 None admin] RBAC: Authorizing `identity:get_user(user_id=68d9ab86ee454d7d9d1e4d3a48048d52)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:10.637190 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3daa4f2-976f-4f08-926b-585d223e7db5 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:10.637349 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692df6c2-6e5e-4bd8-b18a-7972c3323121 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:10.638813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 37/73] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:10 2022] GET /identity/v3/users/68d9ab86ee454d7d9d1e4d3a48048d52 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:10.639948 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3995052-ddbb-4229-b47d-ae274d19bf8e None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:10.640229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3995052-ddbb-4229-b47d-ae274d19bf8e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:10.640497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3995052-ddbb-4229-b47d-ae274d19bf8e None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:10.641726 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 38/74] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:10.642404 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3995052-ddbb-4229-b47d-ae274d19bf8e None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:10.671442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6dbbb-836b-4c63-9873-8a60de666da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:10.672106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6dbbb-836b-4c63-9873-8a60de666da6 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:10.672386 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6dbbb-836b-4c63-9873-8a60de666da6 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:10.672655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6dbbb-836b-4c63-9873-8a60de666da6 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:10.673435 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6dbbb-836b-4c63-9873-8a60de666da6 None admin] RBAC: Adding query filter params (name=member) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:10.673743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6dbbb-836b-4c63-9873-8a60de666da6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:10.675147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6dbbb-836b-4c63-9873-8a60de666da6 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:10.683940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 37/75] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:10.685229 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbdc96-1833-48f7-af85-cd0abc03c629 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:10.685526 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbdc96-1833-48f7-af85-cd0abc03c629 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:10.685802 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbdc96-1833-48f7-af85-cd0abc03c629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:10.702541 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-0dcbdc96-1833-48f7-af85-cd0abc03c629 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:10.704058 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-0dcbdc96-1833-48f7-af85-cd0abc03c629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:10.741046 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3995052-ddbb-4229-b47d-ae274d19bf8e None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:10.767074 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 38/76] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:10.776960 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbdc96-1833-48f7-af85-cd0abc03c629 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:10.782820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3dc617-a4ba-4837-8601-0e116f9d78db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:10.783489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dc617-a4ba-4837-8601-0e116f9d78db None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:10.783776 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dc617-a4ba-4837-8601-0e116f9d78db None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:10.784054 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dc617-a4ba-4837-8601-0e116f9d78db None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:10.784853 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3dc617-a4ba-4837-8601-0e116f9d78db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:10.785147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3dc617-a4ba-4837-8601-0e116f9d78db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:10.786553 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3dc617-a4ba-4837-8601-0e116f9d78db None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:10.794500 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 39/77] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:33:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:10.795683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e3109-86e6-40dd-be6a-bb476818496e None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:10.795973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e3109-86e6-40dd-be6a-bb476818496e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:10.796245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e3109-86e6-40dd-be6a-bb476818496e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:10.811929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-337e3109-86e6-40dd-be6a-bb476818496e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:10.813160 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-337e3109-86e6-40dd-be6a-bb476818496e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:10.864981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbdc96-1833-48f7-af85-cd0abc03c629 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:10.876341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 40/78] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:10.877183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e3109-86e6-40dd-be6a-bb476818496e None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:10.901029 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012696cc-590b-4f48-8b8e-07656c0c1e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:10.901617 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012696cc-590b-4f48-8b8e-07656c0c1e10 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:10.901901 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012696cc-590b-4f48-8b8e-07656c0c1e10 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:10.902175 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012696cc-590b-4f48-8b8e-07656c0c1e10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:10.964831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012696cc-590b-4f48-8b8e-07656c0c1e10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:10.966295 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012696cc-590b-4f48-8b8e-07656c0c1e10 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:10.968271 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e3109-86e6-40dd-be6a-bb476818496e None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:11.000619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff4c7b-8127-4854-90df-afa6dd6b32d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:11.001505 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff4c7b-8127-4854-90df-afa6dd6b32d3 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:11.001778 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff4c7b-8127-4854-90df-afa6dd6b32d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:11.002048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff4c7b-8127-4854-90df-afa6dd6b32d3 None admin] PATH_INFO: `/v3/users/7683dcda9e5649fc8b526a0067af2d8b` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:11.016627 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff4c7b-8127-4854-90df-afa6dd6b32d3 None admin] RBAC: Authorizing `identity:get_user(user_id=7683dcda9e5649fc8b526a0067af2d8b)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:11.018115 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff4c7b-8127-4854-90df-afa6dd6b32d3 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:11.019762 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 41/79] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:10 2022] GET /identity/v3/users/7683dcda9e5649fc8b526a0067af2d8b => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:11.020503 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff4c7b-8127-4854-90df-afa6dd6b32d3 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:11.062253 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 39/80] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:11.063110 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012696cc-590b-4f48-8b8e-07656c0c1e10 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:11.118342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff4c7b-8127-4854-90df-afa6dd6b32d3 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:11.149564 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9b8b9-009f-4f2a-b720-6618bd8ee412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:11.150356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9b8b9-009f-4f2a-b720-6618bd8ee412 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:11.150638 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9b8b9-009f-4f2a-b720-6618bd8ee412 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:11.150913 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9b8b9-009f-4f2a-b720-6618bd8ee412 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:11.160588 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012696cc-590b-4f48-8b8e-07656c0c1e10 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:11.161154 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a9b8b9-009f-4f2a-b720-6618bd8ee412 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:11.168870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-75a9b8b9-009f-4f2a-b720-6618bd8ee412 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:11.168870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:11.168870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:11.168870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:11.168870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:11.168870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:11.168870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:11.168870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:11.168870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:11.168870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:11.168870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:11.168870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:11.168870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:11.168870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:11.168870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:11.168870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:11.169907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:11.169907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:11.169907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:11.169907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:11.169907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:11.169907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:11.169907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:11.169907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:11.169907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:11.169907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:11.169907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:11.169907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:11.169907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:11.169907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:11.169907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:11.169907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:11.170679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:11.170679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:11.170679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:11.170679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:11.170679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:11.170679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:11.170679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:11.170679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:11.170679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:11.170679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:11.170679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:11.170679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:11.170679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 42/81] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:11.171598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9b8b9-009f-4f2a-b720-6618bd8ee412 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:11.198471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e017e2-fdd4-4c27-be84-1469c8eb1d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:11.199353 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e017e2-fdd4-4c27-be84-1469c8eb1d07 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:11.199609 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e017e2-fdd4-4c27-be84-1469c8eb1d07 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:11.199899 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e017e2-fdd4-4c27-be84-1469c8eb1d07 None admin] PATH_INFO: `/v3/users/7683dcda9e5649fc8b526a0067af2d8b` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:11.201138 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e017e2-fdd4-4c27-be84-1469c8eb1d07 None admin] RBAC: Authorizing `identity:get_user(user_id=7683dcda9e5649fc8b526a0067af2d8b)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:11.202614 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e017e2-fdd4-4c27-be84-1469c8eb1d07 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:11.204247 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 40/82] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:11 2022] GET /identity/v3/users/7683dcda9e5649fc8b526a0067af2d8b => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:11.205018 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e017e2-fdd4-4c27-be84-1469c8eb1d07 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:11.269355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9b8b9-009f-4f2a-b720-6618bd8ee412 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:11.300720 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279adef6-901c-4fe9-b998-504305f8ab75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:11.301539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279adef6-901c-4fe9-b998-504305f8ab75 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:11.301822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279adef6-901c-4fe9-b998-504305f8ab75 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:11.302096 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279adef6-901c-4fe9-b998-504305f8ab75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:11.302612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e017e2-fdd4-4c27-be84-1469c8eb1d07 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:11.303295 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279adef6-901c-4fe9-b998-504305f8ab75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:11.304801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279adef6-901c-4fe9-b998-504305f8ab75 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:11.306427 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 43/83] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:33:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:11.307290 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279adef6-901c-4fe9-b998-504305f8ab75 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:11.335777 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512926c-eca1-4602-a80a-6e3313cff84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:11.336416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512926c-eca1-4602-a80a-6e3313cff84c None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:11.336705 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512926c-eca1-4602-a80a-6e3313cff84c None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:11.337003 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512926c-eca1-4602-a80a-6e3313cff84c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:11.398453 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279adef6-901c-4fe9-b998-504305f8ab75 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:11.399469 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512926c-eca1-4602-a80a-6e3313cff84c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:11.400963 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512926c-eca1-4602-a80a-6e3313cff84c None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:11.429572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce4315-f8e9-4f8f-819f-831229c1c87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:11.430470 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce4315-f8e9-4f8f-819f-831229c1c87e None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:11.430814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce4315-f8e9-4f8f-819f-831229c1c87e None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:11.431173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce4315-f8e9-4f8f-819f-831229c1c87e None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:11.432460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce4315-f8e9-4f8f-819f-831229c1c87e None admin] RBAC: Authorizing `identity:get_project(project_id=9ec214e8e5724ce6b35fe31337cb0380)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:11.434027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce4315-f8e9-4f8f-819f-831229c1c87e None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:11.435730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 44/84] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:11 2022] GET /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380 => generated 298 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:11.436471 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce4315-f8e9-4f8f-819f-831229c1c87e None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:11.495408 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 41/85] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:11.497518 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512926c-eca1-4602-a80a-6e3313cff84c None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:11.532151 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce4315-f8e9-4f8f-819f-831229c1c87e None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:11.566357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d53bde-1497-434b-8e80-a945f48f23e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:11.567033 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d53bde-1497-434b-8e80-a945f48f23e2 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:11.567313 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d53bde-1497-434b-8e80-a945f48f23e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:11.567587 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d53bde-1497-434b-8e80-a945f48f23e2 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:11.568339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d53bde-1497-434b-8e80-a945f48f23e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:11.568643 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d53bde-1497-434b-8e80-a945f48f23e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:11.570074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d53bde-1497-434b-8e80-a945f48f23e2 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:11.578355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 45/86] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:11.579019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d53bde-1497-434b-8e80-a945f48f23e2 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:11.590994 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512926c-eca1-4602-a80a-6e3313cff84c None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:11.623839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd7b95-6909-4ecb-af6c-30302dc7beba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:11.624537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd7b95-6909-4ecb-af6c-30302dc7beba None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:11.624903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd7b95-6909-4ecb-af6c-30302dc7beba None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:11.625166 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd7b95-6909-4ecb-af6c-30302dc7beba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:11.673488 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d53bde-1497-434b-8e80-a945f48f23e2 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:11.688328 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd7b95-6909-4ecb-af6c-30302dc7beba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:11.689801 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd7b95-6909-4ecb-af6c-30302dc7beba None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:11.704166 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3a26f-7bb4-4ff7-b54c-3617967ccec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:11.704887 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3a26f-7bb4-4ff7-b54c-3617967ccec3 None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:11.705159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3a26f-7bb4-4ff7-b54c-3617967ccec3 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:11.705427 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3a26f-7bb4-4ff7-b54c-3617967ccec3 None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:11.706127 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3a26f-7bb4-4ff7-b54c-3617967ccec3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:11.707595 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3a26f-7bb4-4ff7-b54c-3617967ccec3 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:11.726536 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 46/87] 10.222.0.26 () {60 vars in 1132 bytes} [Mon May 9 09:33:11 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:33:11.727318 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3a26f-7bb4-4ff7-b54c-3617967ccec3 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:11.802546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 42/88] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:11.806919 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd7b95-6909-4ecb-af6c-30302dc7beba None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:11.830787 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3a26f-7bb4-4ff7-b54c-3617967ccec3 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:11.864487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bcabe-1b96-499d-8672-6ab846d1af3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:11.865209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bcabe-1b96-499d-8672-6ab846d1af3a None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:11.865488 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bcabe-1b96-499d-8672-6ab846d1af3a None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:11.865758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bcabe-1b96-499d-8672-6ab846d1af3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:11.866600 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bcabe-1b96-499d-8672-6ab846d1af3a None admin] RBAC: Adding query filter params (role.id=70ac549382b14b2cbb8b35e5dfc648d4, scope.project.id=9ec214e8e5724ce6b35fe31337cb0380, user.id=68d9ab86ee454d7d9d1e4d3a48048d52) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:11.866912 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bcabe-1b96-499d-8672-6ab846d1af3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:11.868373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bcabe-1b96-499d-8672-6ab846d1af3a None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:11.877522 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 47/89] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:33:11 2022] GET /identity/v3/role_assignments?user.id=68d9ab86ee454d7d9d1e4d3a48048d52&scope.project.id=9ec214e8e5724ce6b35fe31337cb0380&role.id=70ac549382b14b2cbb8b35e5dfc648d4 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:11.878206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bcabe-1b96-499d-8672-6ab846d1af3a None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:11.918912 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd7b95-6909-4ecb-af6c-30302dc7beba None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:11.954044 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051d24c-7308-452d-83a2-01a48d905ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:11.954848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051d24c-7308-452d-83a2-01a48d905ae9 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:11.955129 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051d24c-7308-452d-83a2-01a48d905ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:11.955407 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051d24c-7308-452d-83a2-01a48d905ae9 None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:11.956730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2051d24c-7308-452d-83a2-01a48d905ae9 None admin] RBAC: Authorizing `identity:get_project(project_id=9ec214e8e5724ce6b35fe31337cb0380)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:11.958191 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2051d24c-7308-452d-83a2-01a48d905ae9 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:11.960012 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 43/90] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:11 2022] GET /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:11.961518 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051d24c-7308-452d-83a2-01a48d905ae9 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:11.993586 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bcabe-1b96-499d-8672-6ab846d1af3a None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:12.048952 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e06063-e118-4119-b2a4-790bcde1e905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:12.049863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e06063-e118-4119-b2a4-790bcde1e905 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:12.050138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e06063-e118-4119-b2a4-790bcde1e905 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:12.050405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e06063-e118-4119-b2a4-790bcde1e905 None admin] PATH_INFO: `/v3/users/7683dcda9e5649fc8b526a0067af2d8b` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:12.052333 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e06063-e118-4119-b2a4-790bcde1e905 None admin] RBAC: Authorizing `identity:get_user(user_id=7683dcda9e5649fc8b526a0067af2d8b)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:12.053913 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e06063-e118-4119-b2a4-790bcde1e905 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:12.055824 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 48/91] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:11 2022] GET /identity/v3/users/7683dcda9e5649fc8b526a0067af2d8b => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:12.056436 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e06063-e118-4119-b2a4-790bcde1e905 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:12.108358 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051d24c-7308-452d-83a2-01a48d905ae9 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:12.153310 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648dc49a-e5a7-4aed-975c-30041fc6bb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:12.154177 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648dc49a-e5a7-4aed-975c-30041fc6bb49 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:12.154453 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648dc49a-e5a7-4aed-975c-30041fc6bb49 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:12.154729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648dc49a-e5a7-4aed-975c-30041fc6bb49 None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:12.155941 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648dc49a-e5a7-4aed-975c-30041fc6bb49 None admin] RBAC: Authorizing `identity:get_project(project_id=9ec214e8e5724ce6b35fe31337cb0380)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:12.157497 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648dc49a-e5a7-4aed-975c-30041fc6bb49 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:12.159585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 44/92] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:11 2022] GET /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:12.160363 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648dc49a-e5a7-4aed-975c-30041fc6bb49 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:12.238151 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e06063-e118-4119-b2a4-790bcde1e905 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:12.271202 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648dc49a-e5a7-4aed-975c-30041fc6bb49 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:12.307867 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a628f96-b822-4ccc-9e34-c2696579660b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:12.308607 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a628f96-b822-4ccc-9e34-c2696579660b None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:12.308914 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a628f96-b822-4ccc-9e34-c2696579660b None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:12.309188 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a628f96-b822-4ccc-9e34-c2696579660b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:12.310042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a628f96-b822-4ccc-9e34-c2696579660b None admin] RBAC: Adding query filter params (role.id=fec647282f5a42ab8490975708efad14, scope.project.id=9ec214e8e5724ce6b35fe31337cb0380, user.id=7683dcda9e5649fc8b526a0067af2d8b) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:12.310359 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a628f96-b822-4ccc-9e34-c2696579660b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:12.311884 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a628f96-b822-4ccc-9e34-c2696579660b None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:12.321825 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 49/93] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:33:12 2022] GET /identity/v3/role_assignments?user.id=7683dcda9e5649fc8b526a0067af2d8b&scope.project.id=9ec214e8e5724ce6b35fe31337cb0380&role.id=fec647282f5a42ab8490975708efad14 => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:12.322670 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a628f96-b822-4ccc-9e34-c2696579660b None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:12.341206 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c081ab-cc06-4f96-ac7a-9758aaaa8a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:12.341910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c081ab-cc06-4f96-ac7a-9758aaaa8a64 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:12.342189 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c081ab-cc06-4f96-ac7a-9758aaaa8a64 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:12.342460 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c081ab-cc06-4f96-ac7a-9758aaaa8a64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:12.451168 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c081ab-cc06-4f96-ac7a-9758aaaa8a64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:12.452695 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c081ab-cc06-4f96-ac7a-9758aaaa8a64 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:12.496654 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a628f96-b822-4ccc-9e34-c2696579660b None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:12.559050 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63025c3-0df3-4bd5-aef6-8c939b466b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:12.559723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63025c3-0df3-4bd5-aef6-8c939b466b02 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:12.560329 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63025c3-0df3-4bd5-aef6-8c939b466b02 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:12.560329 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63025c3-0df3-4bd5-aef6-8c939b466b02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:12.561217 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63025c3-0df3-4bd5-aef6-8c939b466b02 None admin] RBAC: Adding query filter params (role.id=0e3ae1b3fdce41c4a42a415efad3b84d, scope.project.id=9ec214e8e5724ce6b35fe31337cb0380, user.id=7683dcda9e5649fc8b526a0067af2d8b) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:12.561674 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63025c3-0df3-4bd5-aef6-8c939b466b02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:12.563915 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63025c3-0df3-4bd5-aef6-8c939b466b02 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:12.573860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 50/94] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:33:12 2022] GET /identity/v3/role_assignments?user.id=7683dcda9e5649fc8b526a0067af2d8b&scope.project.id=9ec214e8e5724ce6b35fe31337cb0380&role.id=0e3ae1b3fdce41c4a42a415efad3b84d => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:12.599331 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 45/95] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:12.605209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63025c3-0df3-4bd5-aef6-8c939b466b02 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:12.731853 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63025c3-0df3-4bd5-aef6-8c939b466b02 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:12.778906 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004b905-90fd-4b1d-9311-32d772dbfde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:12.779732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004b905-90fd-4b1d-9311-32d772dbfde2 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:12.780010 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004b905-90fd-4b1d-9311-32d772dbfde2 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:12.780282 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004b905-90fd-4b1d-9311-32d772dbfde2 None admin] PATH_INFO: `/v3/projects/c749c5092ab543298cf4557399edce98` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:12.785576 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004b905-90fd-4b1d-9311-32d772dbfde2 None admin] RBAC: Authorizing `identity:get_project(project_id=c749c5092ab543298cf4557399edce98)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:12.787217 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004b905-90fd-4b1d-9311-32d772dbfde2 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:12.789097 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 51/96] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:12 2022] GET /identity/v3/projects/c749c5092ab543298cf4557399edce98 => generated 312 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:12.795141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c081ab-cc06-4f96-ac7a-9758aaaa8a64 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:12.895402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c081ab-cc06-4f96-ac7a-9758aaaa8a64 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:12.929470 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115a35b-3042-434b-a65e-0522ecedf266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:12.930234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115a35b-3042-434b-a65e-0522ecedf266 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:12.930417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115a35b-3042-434b-a65e-0522ecedf266 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:12.930674 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115a35b-3042-434b-a65e-0522ecedf266 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:12.931515 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115a35b-3042-434b-a65e-0522ecedf266 None admin] RBAC: Adding query filter params (role.id=fec647282f5a42ab8490975708efad14, scope.project.id=c749c5092ab543298cf4557399edce98, user.id=7683dcda9e5649fc8b526a0067af2d8b) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:12.931823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115a35b-3042-434b-a65e-0522ecedf266 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:12.933350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115a35b-3042-434b-a65e-0522ecedf266 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:12.939207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 46/97] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:33:12 2022] GET /identity/v3/role_assignments?user.id=7683dcda9e5649fc8b526a0067af2d8b&scope.project.id=c749c5092ab543298cf4557399edce98&role.id=fec647282f5a42ab8490975708efad14 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:15.001744 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91927d9-bd88-44ff-bdc4-e4d8fd5b3ec4 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:15.002724 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91927d9-bd88-44ff-bdc4-e4d8fd5b3ec4 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:15.003229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91927d9-bd88-44ff-bdc4-e4d8fd5b3ec4 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:15.004835 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 52/98] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:15.010262 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd3f47-66ae-48bd-91ef-73b962a7a057 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:15.010670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd3f47-66ae-48bd-91ef-73b962a7a057 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:15.010839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd3f47-66ae-48bd-91ef-73b962a7a057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:15.033672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-32cd3f47-66ae-48bd-91ef-73b962a7a057 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:15.034869 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-32cd3f47-66ae-48bd-91ef-73b962a7a057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:15.060097 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb423043-5c93-488f-920a-c7e377601382 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:15.060759 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb423043-5c93-488f-920a-c7e377601382 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:15.061230 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb423043-5c93-488f-920a-c7e377601382 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:15.062791 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 53/99] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:15.067804 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a524904-8d06-4b40-b155-c16fe83cf808 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:15.068335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a524904-8d06-4b40-b155-c16fe83cf808 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:15.068836 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a524904-8d06-4b40-b155-c16fe83cf808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:15.094894 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-9a524904-8d06-4b40-b155-c16fe83cf808 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:15.096432 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-9a524904-8d06-4b40-b155-c16fe83cf808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:15.110852 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 47/100] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:15.134221 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd3f47-66ae-48bd-91ef-73b962a7a057 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:15.195334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 54/101] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:15.235473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a524904-8d06-4b40-b155-c16fe83cf808 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:15.237516 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd3f47-66ae-48bd-91ef-73b962a7a057 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:15.276223 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59ae72-e5a0-48d5-80cb-a958dff34323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:15.277044 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59ae72-e5a0-48d5-80cb-a958dff34323 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:15.277328 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59ae72-e5a0-48d5-80cb-a958dff34323 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:15.277600 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59ae72-e5a0-48d5-80cb-a958dff34323 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:15.288882 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59ae72-e5a0-48d5-80cb-a958dff34323 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:15.296928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-1a59ae72-e5a0-48d5-80cb-a958dff34323 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 09:33:15.296928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:15.296928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:15.296928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:15.296928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:15.296928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:15.296928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:15.296928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:15.296928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:15.296928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:15.296928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:15.296928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:15.296928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:15.296928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:15.296928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:15.296928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:15.297895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:15.297895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:15.297895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:15.297895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:15.297895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:15.297895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:15.297895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:15.297895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:15.297895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:15.297895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:15.297895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:15.297895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:33:15.297895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:15.297895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:15.297895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:15.297895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:15.298665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:15.298665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:15.298665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:15.298665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:15.298665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:15.298665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:15.298665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:15.298665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:15.298665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:15.298665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:15.298665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 09:33:15.298665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:33:15.298665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 48/102] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:15.304692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59ae72-e5a0-48d5-80cb-a958dff34323 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:15.385102 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a524904-8d06-4b40-b155-c16fe83cf808 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:15.405766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59ae72-e5a0-48d5-80cb-a958dff34323 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:15.438823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72e305-4fcd-46b1-870c-ad2d4f46beba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:15.439495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72e305-4fcd-46b1-870c-ad2d4f46beba None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:15.439775 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72e305-4fcd-46b1-870c-ad2d4f46beba None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:15.440048 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72e305-4fcd-46b1-870c-ad2d4f46beba None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:15.440831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d72e305-4fcd-46b1-870c-ad2d4f46beba None admin] RBAC: Adding query filter params (name=reader) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:15.441137 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d72e305-4fcd-46b1-870c-ad2d4f46beba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:15.441401 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d7122-c098-4b37-b539-4ac5761095e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:15.442408 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d7122-c098-4b37-b539-4ac5761095e3 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:15.442519 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d72e305-4fcd-46b1-870c-ad2d4f46beba None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:15.442974 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d7122-c098-4b37-b539-4ac5761095e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:15.443429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d7122-c098-4b37-b539-4ac5761095e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:15.451215 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 49/103] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:15.456732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72e305-4fcd-46b1-870c-ad2d4f46beba None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:15.466417 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d7122-c098-4b37-b539-4ac5761095e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:15.474713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-ef3d7122-c098-4b37-b539-4ac5761095e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:15.474713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:15.474713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:15.474713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:15.474713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:15.474713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:15.474713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:15.474713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:15.474713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:15.474713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:15.474713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:15.474713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:15.474713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:15.474713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:15.474713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:15.474713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:15.475624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:15.475624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:15.475624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:15.475624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:15.475624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:15.475624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:15.475624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:15.475624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:15.475624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:15.475624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:15.475624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:15.475624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:15.475624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:15.475624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:15.475624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:15.475624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:15.476407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:15.476407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:15.476407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:15.476407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:15.476407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:15.476407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:15.476407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:15.476407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:15.476407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:15.476407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:15.476407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:15.476407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:15.479026 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 55/104] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:33:15.484997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d7122-c098-4b37-b539-4ac5761095e3 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:15.559202 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72e305-4fcd-46b1-870c-ad2d4f46beba None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:15.592380 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31059f73-c9a9-40f3-ad85-e38daa6c8111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:15.593279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31059f73-c9a9-40f3-ad85-e38daa6c8111 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:15.593760 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31059f73-c9a9-40f3-ad85-e38daa6c8111 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:15.594243 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31059f73-c9a9-40f3-ad85-e38daa6c8111 None admin] PATH_INFO: `/v3/users/5943d8d8fa4848bfb1ff5fc5be6b6388` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:15.608306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31059f73-c9a9-40f3-ad85-e38daa6c8111 None admin] RBAC: Authorizing `identity:get_user(user_id=5943d8d8fa4848bfb1ff5fc5be6b6388)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:15.610175 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31059f73-c9a9-40f3-ad85-e38daa6c8111 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:15.612125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 50/105] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:15 2022] GET /identity/v3/users/5943d8d8fa4848bfb1ff5fc5be6b6388 => generated 289 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:15.618299 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31059f73-c9a9-40f3-ad85-e38daa6c8111 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:15.644506 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d7122-c098-4b37-b539-4ac5761095e3 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:15.693811 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b18644-404c-4871-b21d-dd9bd3fc049c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:15.694488 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b18644-404c-4871-b21d-dd9bd3fc049c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:15.694767 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b18644-404c-4871-b21d-dd9bd3fc049c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:15.695042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b18644-404c-4871-b21d-dd9bd3fc049c None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:15.695914 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b18644-404c-4871-b21d-dd9bd3fc049c None admin] RBAC: Adding query filter params (name=member) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:15.696087 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b18644-404c-4871-b21d-dd9bd3fc049c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:15.697585 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b18644-404c-4871-b21d-dd9bd3fc049c None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:15.706630 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 56/106] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:15.707950 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff43a30-3536-48dd-8c69-3515397ba5b6 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:15.708233 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff43a30-3536-48dd-8c69-3515397ba5b6 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:15.708506 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff43a30-3536-48dd-8c69-3515397ba5b6 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:15.710261 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 57/107] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:15.710981 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab26da-37b5-44f3-954e-f314a5e6a195 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:15.711260 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab26da-37b5-44f3-954e-f314a5e6a195 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:15.711534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab26da-37b5-44f3-954e-f314a5e6a195 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:15.712756 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 58/108] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:15.713444 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab26da-37b5-44f3-954e-f314a5e6a195 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:15.723372 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31059f73-c9a9-40f3-ad85-e38daa6c8111 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:15.760696 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3362fd-2047-4fcb-974e-c16185c89568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:15.760696 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3362fd-2047-4fcb-974e-c16185c89568 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:15.760696 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3362fd-2047-4fcb-974e-c16185c89568 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:15.760696 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3362fd-2047-4fcb-974e-c16185c89568 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:15.821374 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3362fd-2047-4fcb-974e-c16185c89568 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:15.821687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab26da-37b5-44f3-954e-f314a5e6a195 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:15.822765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3362fd-2047-4fcb-974e-c16185c89568 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:15.856085 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4cfc26-d939-4d89-b371-925310ee6af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:15.856981 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cfc26-d939-4d89-b371-925310ee6af4 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:15.857258 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cfc26-d939-4d89-b371-925310ee6af4 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:15.857544 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cfc26-d939-4d89-b371-925310ee6af4 None admin] PATH_INFO: `/v3/users/7683dcda9e5649fc8b526a0067af2d8b` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:15.858740 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4cfc26-d939-4d89-b371-925310ee6af4 None admin] RBAC: Authorizing `identity:get_user(user_id=7683dcda9e5649fc8b526a0067af2d8b)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:15.860204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4cfc26-d939-4d89-b371-925310ee6af4 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:15.866482 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 59/109] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:15 2022] GET /identity/v3/users/7683dcda9e5649fc8b526a0067af2d8b => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:15.866482 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dcc2b-f1b7-4ee2-ad02-a19142b8a6a4 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:15.866482 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dcc2b-f1b7-4ee2-ad02-a19142b8a6a4 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:15.866482 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dcc2b-f1b7-4ee2-ad02-a19142b8a6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:15.882106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-562dcc2b-f1b7-4ee2-ad02-a19142b8a6a4 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:15.883286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-562dcc2b-f1b7-4ee2-ad02-a19142b8a6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:15.924882 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 51/110] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:15.924882 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31a621-a855-4ab4-a62c-72925e345284 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:15.924882 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31a621-a855-4ab4-a62c-72925e345284 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:15.924882 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31a621-a855-4ab4-a62c-72925e345284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:15.941005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-5d31a621-a855-4ab4-a62c-72925e345284 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:15.942170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-5d31a621-a855-4ab4-a62c-72925e345284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:15.960682 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 60/111] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:15.960682 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57998da0-d174-4f89-a94a-c960c8f48223 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:15.960682 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57998da0-d174-4f89-a94a-c960c8f48223 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:15.960682 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57998da0-d174-4f89-a94a-c960c8f48223 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:15.960682 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 61/112] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:15.960682 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57998da0-d174-4f89-a94a-c960c8f48223 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:16.012866 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 52/113] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:16.013571 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31a621-a855-4ab4-a62c-72925e345284 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:16.066149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57998da0-d174-4f89-a94a-c960c8f48223 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:16.099322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24cbd2-6c97-4ca8-86f7-8f4a9c5f3f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:16.100109 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24cbd2-6c97-4ca8-86f7-8f4a9c5f3f8d None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:16.100390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24cbd2-6c97-4ca8-86f7-8f4a9c5f3f8d None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:16.100694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24cbd2-6c97-4ca8-86f7-8f4a9c5f3f8d None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:16.101940 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e24cbd2-6c97-4ca8-86f7-8f4a9c5f3f8d None admin] RBAC: Authorizing `identity:get_project(project_id=9ec214e8e5724ce6b35fe31337cb0380)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:16.103414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e24cbd2-6c97-4ca8-86f7-8f4a9c5f3f8d None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:16.108209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31a621-a855-4ab4-a62c-72925e345284 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:16.110020 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 62/114] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:15 2022] GET /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:16.111310 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d8f74-687d-43ad-8126-bce42fbf8915 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:16.111597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d8f74-687d-43ad-8126-bce42fbf8915 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:16.111878 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d8f74-687d-43ad-8126-bce42fbf8915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:16.128046 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-296d8f74-687d-43ad-8126-bce42fbf8915 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:16.129265 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-296d8f74-687d-43ad-8126-bce42fbf8915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:16.141978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad845ccd-288b-49e0-a77e-99c33166c39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:16.142840 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad845ccd-288b-49e0-a77e-99c33166c39a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:16.143113 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad845ccd-288b-49e0-a77e-99c33166c39a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:16.143379 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad845ccd-288b-49e0-a77e-99c33166c39a None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:16.144538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad845ccd-288b-49e0-a77e-99c33166c39a None admin] RBAC: Authorizing `identity:get_project(project_id=9ec214e8e5724ce6b35fe31337cb0380)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:16.146018 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad845ccd-288b-49e0-a77e-99c33166c39a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:16.152584 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 53/115] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:16 2022] GET /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:16.153282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad845ccd-288b-49e0-a77e-99c33166c39a None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:16.192156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 63/116] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:16.193034 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d8f74-687d-43ad-8126-bce42fbf8915 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:16.259910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad845ccd-288b-49e0-a77e-99c33166c39a None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:16.292620 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d8f74-687d-43ad-8126-bce42fbf8915 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:16.300375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2333b-4c60-4a10-a0af-7d0da0559e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:16.301214 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2333b-4c60-4a10-a0af-7d0da0559e99 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:16.301498 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2333b-4c60-4a10-a0af-7d0da0559e99 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:16.301769 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2333b-4c60-4a10-a0af-7d0da0559e99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:16.316963 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2333b-4c60-4a10-a0af-7d0da0559e99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:16.324587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-b5f2333b-4c60-4a10-a0af-7d0da0559e99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:33:16.324587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:16.324587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:16.324587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:16.324587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:16.324587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:16.324587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:16.324587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:16.324587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:16.324587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:16.324587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:16.324587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:16.324587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:16.324587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:16.324587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:16.324587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:16.325509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:16.325509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:16.325509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:16.325509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:16.325509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:16.325509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:16.325509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:16.325509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:16.325509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:16.325509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:16.325509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:16.325509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:33:16.325509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:16.325509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:16.325509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:16.325509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:16.326301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:16.326301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:16.326301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:16.326301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:16.326301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:16.326301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:16.326301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:16.326301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:16.326301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:16.326301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:16.326301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:33:16.326301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:33:16.326301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 54/117] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:33:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:16.326966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2333b-4c60-4a10-a0af-7d0da0559e99 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:16.333767 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b02184-48a6-4c71-82ce-f40ca624f1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:16.334575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b02184-48a6-4c71-82ce-f40ca624f1f0 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:16.334853 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b02184-48a6-4c71-82ce-f40ca624f1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:16.335127 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b02184-48a6-4c71-82ce-f40ca624f1f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:16.336286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b02184-48a6-4c71-82ce-f40ca624f1f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:16.337845 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b02184-48a6-4c71-82ce-f40ca624f1f0 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:16.344249 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 64/118] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:33:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:16.344249 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b02184-48a6-4c71-82ce-f40ca624f1f0 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:16.429685 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2333b-4c60-4a10-a0af-7d0da0559e99 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:16.435434 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b02184-48a6-4c71-82ce-f40ca624f1f0 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:16.461732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f954f-9413-42cf-b8e6-18141eec78cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:16.462627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f954f-9413-42cf-b8e6-18141eec78cf None admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:16.462914 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f954f-9413-42cf-b8e6-18141eec78cf None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:16.463189 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f954f-9413-42cf-b8e6-18141eec78cf None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380/users/7683dcda9e5649fc8b526a0067af2d8b/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:16.465292 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f954f-9413-42cf-b8e6-18141eec78cf None admin] RBAC: Authorizing `identity:create_grant(project_id=9ec214e8e5724ce6b35fe31337cb0380, user_id=7683dcda9e5649fc8b526a0067af2d8b, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:16.466811 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f954f-9413-42cf-b8e6-18141eec78cf None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:16.467797 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954801d5-e389-4b7c-b58b-b24d87aee2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:16.468443 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954801d5-e389-4b7c-b58b-b24d87aee2dc None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:16.468747 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954801d5-e389-4b7c-b58b-b24d87aee2dc None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:16.469022 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954801d5-e389-4b7c-b58b-b24d87aee2dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:16.469846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954801d5-e389-4b7c-b58b-b24d87aee2dc None admin] RBAC: Adding query filter params (role.id=78ec5dc2a67d445389f9a7a4458a12cb, scope.project.id=9ec214e8e5724ce6b35fe31337cb0380, user.id=5943d8d8fa4848bfb1ff5fc5be6b6388) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:16.470151 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954801d5-e389-4b7c-b58b-b24d87aee2dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:16.471581 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954801d5-e389-4b7c-b58b-b24d87aee2dc None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:16.488714 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 65/119] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:33:16 2022] GET /identity/v3/role_assignments?user.id=5943d8d8fa4848bfb1ff5fc5be6b6388&scope.project.id=9ec214e8e5724ce6b35fe31337cb0380&role.id=78ec5dc2a67d445389f9a7a4458a12cb => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:16.488714 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954801d5-e389-4b7c-b58b-b24d87aee2dc None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:16.497607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 55/120] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:33:16 2022] PUT /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380/users/7683dcda9e5649fc8b526a0067af2d8b/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:33:16.498249 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f954f-9413-42cf-b8e6-18141eec78cf None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:16.612292 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954801d5-e389-4b7c-b58b-b24d87aee2dc None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:16.644268 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f954f-9413-42cf-b8e6-18141eec78cf None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:16.654568 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70427294-7169-471e-8e97-93f4ed82e6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:16.655549 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70427294-7169-471e-8e97-93f4ed82e6d3 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:16.655641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70427294-7169-471e-8e97-93f4ed82e6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:16.655910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70427294-7169-471e-8e97-93f4ed82e6d3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:16.668132 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70427294-7169-471e-8e97-93f4ed82e6d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:16.676392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-70427294-7169-471e-8e97-93f4ed82e6d3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:16.676392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:16.676392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:16.676392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:16.676392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:16.676392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:16.676392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:16.676392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:16.676392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:16.676392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:16.676392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:16.676392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:16.676392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:16.676392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:16.676392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:16.676392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:16.677758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:16.677758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:16.677758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:16.677758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:16.677758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:16.677758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:16.677758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:16.677758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:16.677758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:16.677758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:16.677758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:16.677758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:16.677758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:16.677758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:16.677758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:16.677758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:16.678793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:16.678793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:16.678793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:16.678793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:16.678793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:16.678793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:16.678793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:16.678793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:16.678793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:16.678793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:16.678793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:16.678793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:16.678793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 66/121] 10.222.0.26 () {56 vars in 1114 bytes} [Mon May 9 09:33:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:16.678793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70427294-7169-471e-8e97-93f4ed82e6d3 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:16.692391 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6babb79a-1755-499b-ab1f-940f6061877e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:16.693516 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babb79a-1755-499b-ab1f-940f6061877e None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:16.694022 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babb79a-1755-499b-ab1f-940f6061877e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:16.694496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babb79a-1755-499b-ab1f-940f6061877e None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:16.695511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6babb79a-1755-499b-ab1f-940f6061877e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:16.696187 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6babb79a-1755-499b-ab1f-940f6061877e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:16.697862 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6babb79a-1755-499b-ab1f-940f6061877e None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:16.709344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 56/122] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:16.712415 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43966070-6453-4663-9f45-630429a5165e None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:16.712990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43966070-6453-4663-9f45-630429a5165e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:16.714148 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43966070-6453-4663-9f45-630429a5165e None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:16.716624 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 57/123] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:16.717602 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43966070-6453-4663-9f45-630429a5165e None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:16.808488 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70427294-7169-471e-8e97-93f4ed82e6d3 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:16.822709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43966070-6453-4663-9f45-630429a5165e None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:16.849844 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078279c-83c4-4bef-9722-baec5303242a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:16.851148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078279c-83c4-4bef-9722-baec5303242a None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:16.851840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078279c-83c4-4bef-9722-baec5303242a None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:16.852345 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078279c-83c4-4bef-9722-baec5303242a None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:16.853424 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078279c-83c4-4bef-9722-baec5303242a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:16.855501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078279c-83c4-4bef-9722-baec5303242a None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:16.855698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447af1a-e2a5-477d-a1c0-33a2b6dcc8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:16.856367 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447af1a-e2a5-477d-a1c0-33a2b6dcc8b7 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:16.856714 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447af1a-e2a5-477d-a1c0-33a2b6dcc8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:16.856995 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447af1a-e2a5-477d-a1c0-33a2b6dcc8b7 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:16.857966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447af1a-e2a5-477d-a1c0-33a2b6dcc8b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:16.858314 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447af1a-e2a5-477d-a1c0-33a2b6dcc8b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:16.859747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447af1a-e2a5-477d-a1c0-33a2b6dcc8b7 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:16.873332 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 58/124] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:33:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:16.873332 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447af1a-e2a5-477d-a1c0-33a2b6dcc8b7 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:16.893200 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 67/125] 10.222.0.26 () {60 vars in 1141 bytes} [Mon May 9 09:33:16 2022] POST /identity/v3/projects => generated 302 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:33:16.894693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e5998-2b9e-42dc-b618-8a0790970f1b None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:16.895017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e5998-2b9e-42dc-b618-8a0790970f1b None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:16.895341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e5998-2b9e-42dc-b618-8a0790970f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:16.913957 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-507e5998-2b9e-42dc-b618-8a0790970f1b None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:16.915252 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-507e5998-2b9e-42dc-b618-8a0790970f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:16.988873 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447af1a-e2a5-477d-a1c0-33a2b6dcc8b7 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:16.994406 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 68/126] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:16.995385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e5998-2b9e-42dc-b618-8a0790970f1b None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:17.040689 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865fff3-a9bd-40e4-b3ef-f309aee6e8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:17.042356 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865fff3-a9bd-40e4-b3ef-f309aee6e8c8 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:17.042869 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865fff3-a9bd-40e4-b3ef-f309aee6e8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:17.043308 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865fff3-a9bd-40e4-b3ef-f309aee6e8c8 None admin] PATH_INFO: `/v3/users/68d9ab86ee454d7d9d1e4d3a48048d52` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:17.047695 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865fff3-a9bd-40e4-b3ef-f309aee6e8c8 None admin] RBAC: Authorizing `identity:get_user(user_id=68d9ab86ee454d7d9d1e4d3a48048d52)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:17.049240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865fff3-a9bd-40e4-b3ef-f309aee6e8c8 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:17.051174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 59/127] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:16 2022] GET /identity/v3/users/68d9ab86ee454d7d9d1e4d3a48048d52 => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:17.053286 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865fff3-a9bd-40e4-b3ef-f309aee6e8c8 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:17.106637 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e5998-2b9e-42dc-b618-8a0790970f1b None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:17.141753 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcad457-d0d8-4ea7-860d-4d560b533fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:17.143045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcad457-d0d8-4ea7-860d-4d560b533fe5 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:17.143520 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcad457-d0d8-4ea7-860d-4d560b533fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:17.143993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcad457-d0d8-4ea7-860d-4d560b533fe5 None admin] PATH_INFO: `/v3/users/7683dcda9e5649fc8b526a0067af2d8b` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:17.145481 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcad457-d0d8-4ea7-860d-4d560b533fe5 None admin] RBAC: Authorizing `identity:get_user(user_id=7683dcda9e5649fc8b526a0067af2d8b)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:17.147312 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcad457-d0d8-4ea7-860d-4d560b533fe5 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:17.149225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 69/128] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:16 2022] GET /identity/v3/users/7683dcda9e5649fc8b526a0067af2d8b => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:17.150178 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcad457-d0d8-4ea7-860d-4d560b533fe5 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:17.163630 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865fff3-a9bd-40e4-b3ef-f309aee6e8c8 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:17.213512 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456de50-7cc8-4ec3-ab2c-3120a9be16ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:17.214422 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456de50-7cc8-4ec3-ab2c-3120a9be16ab None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:17.214705 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456de50-7cc8-4ec3-ab2c-3120a9be16ab None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:17.214979 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456de50-7cc8-4ec3-ab2c-3120a9be16ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:17.234323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f456de50-7cc8-4ec3-ab2c-3120a9be16ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:17.250383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-f456de50-7cc8-4ec3-ab2c-3120a9be16ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:17.250383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:17.250383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:17.250383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:17.250383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:17.250383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:17.250383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:17.250383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:17.250383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:17.250383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:17.250383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:17.250383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:17.250383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:17.250383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:17.250383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:17.250383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:17.251311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:17.251311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:17.251311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:17.251311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:17.251311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:17.251311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:17.251311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:17.251311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:17.251311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:17.251311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:17.251311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:17.251311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:33:17.251311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:17.251311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:17.251311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:17.251311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:17.252085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:17.252085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:17.252085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:17.252085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:17.252085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:17.252085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:17.252085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:17.252085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:17.252085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:17.252085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:17.252085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:17.252085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:33:17.252085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 60/129] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:17.253483 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456de50-7cc8-4ec3-ab2c-3120a9be16ab None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:17.264148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcad457-d0d8-4ea7-860d-4d560b533fe5 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:17.299350 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf740f1-fa3b-4490-8c84-a23363aefeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:17.300582 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf740f1-fa3b-4490-8c84-a23363aefeb3 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:17.301016 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf740f1-fa3b-4490-8c84-a23363aefeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:17.301427 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf740f1-fa3b-4490-8c84-a23363aefeb3 None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:17.302800 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf740f1-fa3b-4490-8c84-a23363aefeb3 None admin] RBAC: Authorizing `identity:get_project(project_id=9ec214e8e5724ce6b35fe31337cb0380)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:17.304412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf740f1-fa3b-4490-8c84-a23363aefeb3 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:17.306316 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 70/130] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:17 2022] GET /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:17.307230 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf740f1-fa3b-4490-8c84-a23363aefeb3 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:17.374130 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456de50-7cc8-4ec3-ab2c-3120a9be16ab None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:17.410634 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf740f1-fa3b-4490-8c84-a23363aefeb3 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:17.411448 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4c7a0-e06d-41b7-b51b-7c826e505a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:17.412559 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4c7a0-e06d-41b7-b51b-7c826e505a00 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:17.412993 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4c7a0-e06d-41b7-b51b-7c826e505a00 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:17.413373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4c7a0-e06d-41b7-b51b-7c826e505a00 None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:17.415471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4c7a0-e06d-41b7-b51b-7c826e505a00 None admin] RBAC: Authorizing `identity:get_project(project_id=9ec214e8e5724ce6b35fe31337cb0380)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:17.417067 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4c7a0-e06d-41b7-b51b-7c826e505a00 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:17.418950 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 61/131] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:17 2022] GET /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:17.420245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4c7a0-e06d-41b7-b51b-7c826e505a00 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:17.447925 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bd9c1-5f15-4105-baae-31aa54d0ff1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:17.449009 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bd9c1-5f15-4105-baae-31aa54d0ff1e None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:17.449291 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bd9c1-5f15-4105-baae-31aa54d0ff1e None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:17.449563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bd9c1-5f15-4105-baae-31aa54d0ff1e None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:17.450316 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728bd9c1-5f15-4105-baae-31aa54d0ff1e None admin] RBAC: Adding query filter params (name=member) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:17.450613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728bd9c1-5f15-4105-baae-31aa54d0ff1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:17.452012 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728bd9c1-5f15-4105-baae-31aa54d0ff1e None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:17.461184 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 71/132] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:17.461980 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bd9c1-5f15-4105-baae-31aa54d0ff1e None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:17.544061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4c7a0-e06d-41b7-b51b-7c826e505a00 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:17.571024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bd9c1-5f15-4105-baae-31aa54d0ff1e None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:17.586382 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c35e7c-d5d0-4c66-827f-5f59a3355767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:17.588016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c35e7c-d5d0-4c66-827f-5f59a3355767 None admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:17.588451 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c35e7c-d5d0-4c66-827f-5f59a3355767 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:17.589153 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c35e7c-d5d0-4c66-827f-5f59a3355767 None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380/users/68d9ab86ee454d7d9d1e4d3a48048d52/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:17.591718 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c35e7c-d5d0-4c66-827f-5f59a3355767 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ec214e8e5724ce6b35fe31337cb0380, user_id=68d9ab86ee454d7d9d1e4d3a48048d52, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:17.593360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c35e7c-d5d0-4c66-827f-5f59a3355767 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:17.611508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 62/133] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:33:17 2022] PUT /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380/users/68d9ab86ee454d7d9d1e4d3a48048d52/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:33:17.616695 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c35e7c-d5d0-4c66-827f-5f59a3355767 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:17.635611 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5124c-7913-4601-885b-0d00eada3ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:17.636651 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5124c-7913-4601-885b-0d00eada3ed0 None admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:17.636996 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5124c-7913-4601-885b-0d00eada3ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:17.637270 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5124c-7913-4601-885b-0d00eada3ed0 None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380/users/7683dcda9e5649fc8b526a0067af2d8b/roles/0e3ae1b3fdce41c4a42a415efad3b84d` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:17.653010 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5124c-7913-4601-885b-0d00eada3ed0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ec214e8e5724ce6b35fe31337cb0380, user_id=7683dcda9e5649fc8b526a0067af2d8b, role_id=0e3ae1b3fdce41c4a42a415efad3b84d)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:17.654698 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5124c-7913-4601-885b-0d00eada3ed0 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:17.686967 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 72/134] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:33:17 2022] PUT /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380/users/7683dcda9e5649fc8b526a0067af2d8b/roles/0e3ae1b3fdce41c4a42a415efad3b84d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:33:17.815858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c35e7c-d5d0-4c66-827f-5f59a3355767 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:17.852580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ef926-a0a9-470f-9aa6-98bb268064df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:17.852986 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ef926-a0a9-470f-9aa6-98bb268064df None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:17.853264 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ef926-a0a9-470f-9aa6-98bb268064df None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:17.853533 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ef926-a0a9-470f-9aa6-98bb268064df None admin] PATH_INFO: `/v3/users/7683dcda9e5649fc8b526a0067af2d8b` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:17.855591 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ef926-a0a9-470f-9aa6-98bb268064df None admin] RBAC: Authorizing `identity:get_user(user_id=7683dcda9e5649fc8b526a0067af2d8b)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:17.856267 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ef926-a0a9-470f-9aa6-98bb268064df None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:17.857957 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 63/135] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:17 2022] GET /identity/v3/users/7683dcda9e5649fc8b526a0067af2d8b => generated 275 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:17.863857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5124c-7913-4601-885b-0d00eada3ed0 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:17.969491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5124c-7913-4601-885b-0d00eada3ed0 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:18.004818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7d468-0644-49a6-9ef0-74568be3870e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:18.006087 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7d468-0644-49a6-9ef0-74568be3870e None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:18.006579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7d468-0644-49a6-9ef0-74568be3870e None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:18.007052 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7d468-0644-49a6-9ef0-74568be3870e None admin] PATH_INFO: `/v3/projects/c749c5092ab543298cf4557399edce98` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:18.008502 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7d468-0644-49a6-9ef0-74568be3870e None admin] RBAC: Authorizing `identity:get_project(project_id=c749c5092ab543298cf4557399edce98)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:18.010222 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7d468-0644-49a6-9ef0-74568be3870e None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:18.012319 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 73/136] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:17 2022] GET /identity/v3/projects/c749c5092ab543298cf4557399edce98 => generated 312 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:18.018517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ef926-a0a9-470f-9aa6-98bb268064df None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:18.149280 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ef926-a0a9-470f-9aa6-98bb268064df None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:18.183877 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c16f4e-0961-46ad-95d1-5a8968a72fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:18.184810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c16f4e-0961-46ad-95d1-5a8968a72fa5 None admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:18.185088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c16f4e-0961-46ad-95d1-5a8968a72fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:18.185357 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c16f4e-0961-46ad-95d1-5a8968a72fa5 None admin] PATH_INFO: `/v3/projects/c749c5092ab543298cf4557399edce98/users/7683dcda9e5649fc8b526a0067af2d8b/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:18.187299 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c16f4e-0961-46ad-95d1-5a8968a72fa5 None admin] RBAC: Authorizing `identity:create_grant(project_id=c749c5092ab543298cf4557399edce98, user_id=7683dcda9e5649fc8b526a0067af2d8b, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:18.188785 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c16f4e-0961-46ad-95d1-5a8968a72fa5 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:18.205275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 64/137] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:33:18 2022] PUT /identity/v3/projects/c749c5092ab543298cf4557399edce98/users/7683dcda9e5649fc8b526a0067af2d8b/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:33:19.310280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f94169-7c4e-4f2c-99b7-c28522a864c7 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:19.311358 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f94169-7c4e-4f2c-99b7-c28522a864c7 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:19.311838 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f94169-7c4e-4f2c-99b7-c28522a864c7 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:19.313843 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 74/138] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:19.326826 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa12be-db43-458b-8ebb-81723b86055c None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:19.327376 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa12be-db43-458b-8ebb-81723b86055c None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:19.327376 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa12be-db43-458b-8ebb-81723b86055c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:19.353251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-28aa12be-db43-458b-8ebb-81723b86055c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:19.354440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-28aa12be-db43-458b-8ebb-81723b86055c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:19.440538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 65/139] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:19.472359 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f94169-7c4e-4f2c-99b7-c28522a864c7 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:19.518006 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6eaaa7-f5ab-47b5-8428-f53aa6f75957 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:19.518342 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6eaaa7-f5ab-47b5-8428-f53aa6f75957 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:19.518558 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6eaaa7-f5ab-47b5-8428-f53aa6f75957 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:19.521174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 66/140] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:19.533293 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b26a61-8467-4784-b241-375509bef85b None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:19.533599 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b26a61-8467-4784-b241-375509bef85b None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:19.533841 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b26a61-8467-4784-b241-375509bef85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:19.562049 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-52b26a61-8467-4784-b241-375509bef85b None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:19.563854 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-52b26a61-8467-4784-b241-375509bef85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:19.651961 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f94169-7c4e-4f2c-99b7-c28522a864c7 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:19.667284 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 67/141] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:33:19.699740 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb85359-94f1-4d2e-a525-709786412372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:19.700587 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb85359-94f1-4d2e-a525-709786412372 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:19.700921 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb85359-94f1-4d2e-a525-709786412372 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:19.701190 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb85359-94f1-4d2e-a525-709786412372 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:19.716558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb85359-94f1-4d2e-a525-709786412372 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:19.724280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-7bb85359-94f1-4d2e-a525-709786412372 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 09:33:19.724280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:19.724280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:19.724280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:19.724280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:19.724280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:19.724280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:19.724280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:19.724280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:19.724280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:19.724280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:19.724280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:19.724280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:19.724280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:19.724280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:19.724280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:19.725158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:19.725158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:19.725158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:19.725158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:19.725158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:19.725158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:19.725158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:19.725158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:19.725158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:19.725158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:19.725158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:19.725158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:19.725158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:19.725158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:19.725158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:19.725158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:19.725932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:19.725932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:19.725932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:19.725932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:19.725932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:19.725932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:19.725932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:19.725932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:19.725932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:19.725932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:19.725932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 09:33:19.725932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:19.725932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 75/142] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:19.732509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b26a61-8467-4784-b241-375509bef85b None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:19.770132 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb85359-94f1-4d2e-a525-709786412372 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:19.884774 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb85359-94f1-4d2e-a525-709786412372 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:19.922014 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c3b01-9728-46eb-958a-7bd49c5e7e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:19.922810 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c3b01-9728-46eb-958a-7bd49c5e7e87 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:19.923086 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c3b01-9728-46eb-958a-7bd49c5e7e87 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:19.923355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c3b01-9728-46eb-958a-7bd49c5e7e87 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:19.925891 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b26a61-8467-4784-b241-375509bef85b None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:19.957559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c3b01-9728-46eb-958a-7bd49c5e7e87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:19.967841 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17bccd1-fbdb-4513-9bb8-184d22655cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:19.968340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-fb9c3b01-9728-46eb-958a-7bd49c5e7e87 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:19.968340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:19.968340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:19.968340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:19.968340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:19.968340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:19.968340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:19.968340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:19.968340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:19.968340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:19.968340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:19.968340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:19.968340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:19.968340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:19.968340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:19.968340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:19.969663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:19.969663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:19.969663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:19.969663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:19.969663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:19.969663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:19.969663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:19.969663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:19.969663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:19.969663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:19.969663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:19.969663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:19.969663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:19.969663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:19.969663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:19.969663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:19.970423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:19.970423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:19.970423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:19.970423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:19.970423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:19.970423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:19.970423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:19.970423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:19.970423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:19.970423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:19.970423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:19.970423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:19.971016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bccd1-fbdb-4513-9bb8-184d22655cbe None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:19.971016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bccd1-fbdb-4513-9bb8-184d22655cbe None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:19.971016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bccd1-fbdb-4513-9bb8-184d22655cbe None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:19.971016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17bccd1-fbdb-4513-9bb8-184d22655cbe None admin] RBAC: Adding query filter params (name=reader) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:19.971259 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17bccd1-fbdb-4513-9bb8-184d22655cbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:19.973242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17bccd1-fbdb-4513-9bb8-184d22655cbe None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:19.973707 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 76/143] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:33:19.987025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 68/144] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:19.992557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c3b01-9728-46eb-958a-7bd49c5e7e87 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:19.994151 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17bccd1-fbdb-4513-9bb8-184d22655cbe None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:20.140624 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17bccd1-fbdb-4513-9bb8-184d22655cbe None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:20.189440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dffa8-393f-40b2-93ff-deb16c5f3a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:20.190126 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dffa8-393f-40b2-93ff-deb16c5f3a33 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:20.190405 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dffa8-393f-40b2-93ff-deb16c5f3a33 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:20.190681 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dffa8-393f-40b2-93ff-deb16c5f3a33 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:20.191455 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704dffa8-393f-40b2-93ff-deb16c5f3a33 None admin] RBAC: Adding query filter params (name=member) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:20.191760 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704dffa8-393f-40b2-93ff-deb16c5f3a33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:20.193207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704dffa8-393f-40b2-93ff-deb16c5f3a33 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:20.206844 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 69/145] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:20.212330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dffa8-393f-40b2-93ff-deb16c5f3a33 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:20.214645 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c3b01-9728-46eb-958a-7bd49c5e7e87 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:20.287064 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be06ab9-4e51-45bc-ba28-68340c540a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:20.288008 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be06ab9-4e51-45bc-ba28-68340c540a74 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:20.288287 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be06ab9-4e51-45bc-ba28-68340c540a74 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:20.288559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be06ab9-4e51-45bc-ba28-68340c540a74 None admin] PATH_INFO: `/v3/users/5943d8d8fa4848bfb1ff5fc5be6b6388` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:20.297485 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be06ab9-4e51-45bc-ba28-68340c540a74 None admin] RBAC: Authorizing `identity:get_user(user_id=5943d8d8fa4848bfb1ff5fc5be6b6388)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:20.298987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be06ab9-4e51-45bc-ba28-68340c540a74 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:20.300730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 77/146] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:19 2022] GET /identity/v3/users/5943d8d8fa4848bfb1ff5fc5be6b6388 => generated 289 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:20.305963 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be06ab9-4e51-45bc-ba28-68340c540a74 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:20.433959 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dffa8-393f-40b2-93ff-deb16c5f3a33 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:20.456268 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be06ab9-4e51-45bc-ba28-68340c540a74 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:20.517460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b84f0-6db9-4a59-9a9d-be57088d96dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:20.518334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b84f0-6db9-4a59-9a9d-be57088d96dc None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:20.518619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b84f0-6db9-4a59-9a9d-be57088d96dc None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:20.518897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b84f0-6db9-4a59-9a9d-be57088d96dc None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:20.521641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b84f0-6db9-4a59-9a9d-be57088d96dc None admin] RBAC: Authorizing `identity:get_project(project_id=9ec214e8e5724ce6b35fe31337cb0380)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:20.523159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b84f0-6db9-4a59-9a9d-be57088d96dc None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:20.527780 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 78/147] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:20 2022] GET /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:20.532245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b84f0-6db9-4a59-9a9d-be57088d96dc None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:20.532444 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4451303-3e8b-4878-acaf-9aaa0465cf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:20.533445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4451303-3e8b-4878-acaf-9aaa0465cf19 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:20.533725 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4451303-3e8b-4878-acaf-9aaa0465cf19 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:20.533996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4451303-3e8b-4878-acaf-9aaa0465cf19 None admin] PATH_INFO: `/v3/users/7683dcda9e5649fc8b526a0067af2d8b` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:20.535198 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4451303-3e8b-4878-acaf-9aaa0465cf19 None admin] RBAC: Authorizing `identity:get_user(user_id=7683dcda9e5649fc8b526a0067af2d8b)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:20.536643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4451303-3e8b-4878-acaf-9aaa0465cf19 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:20.538455 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 70/148] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:20 2022] GET /identity/v3/users/7683dcda9e5649fc8b526a0067af2d8b => generated 275 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:20.549635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4451303-3e8b-4878-acaf-9aaa0465cf19 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:20.717045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b84f0-6db9-4a59-9a9d-be57088d96dc None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:20.738242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4451303-3e8b-4878-acaf-9aaa0465cf19 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:20.790160 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53b080-2570-4e41-8b7b-17ebe028cbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:20.791080 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53b080-2570-4e41-8b7b-17ebe028cbc1 None admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:20.791362 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53b080-2570-4e41-8b7b-17ebe028cbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:20.791640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53b080-2570-4e41-8b7b-17ebe028cbc1 None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380/users/5943d8d8fa4848bfb1ff5fc5be6b6388/roles/78ec5dc2a67d445389f9a7a4458a12cb` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:20.796176 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a81078-34b4-46b0-be13-ad9ae2a8c4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:20.796872 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a81078-34b4-46b0-be13-ad9ae2a8c4d4 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:20.797153 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a81078-34b4-46b0-be13-ad9ae2a8c4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:20.797425 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a81078-34b4-46b0-be13-ad9ae2a8c4d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:20.797597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53b080-2570-4e41-8b7b-17ebe028cbc1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ec214e8e5724ce6b35fe31337cb0380, user_id=5943d8d8fa4848bfb1ff5fc5be6b6388, role_id=78ec5dc2a67d445389f9a7a4458a12cb)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:20.799040 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53b080-2570-4e41-8b7b-17ebe028cbc1 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:20.824097 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 79/149] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:33:20 2022] PUT /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380/users/5943d8d8fa4848bfb1ff5fc5be6b6388/roles/78ec5dc2a67d445389f9a7a4458a12cb => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:33:20.888323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a81078-34b4-46b0-be13-ad9ae2a8c4d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:20.889888 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a81078-34b4-46b0-be13-ad9ae2a8c4d4 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:20.898593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b03ab-602e-4739-8554-c5f6d38a5757 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:20.899218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b03ab-602e-4739-8554-c5f6d38a5757 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:20.899725 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b03ab-602e-4739-8554-c5f6d38a5757 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:20.905529 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 80/150] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:20.907100 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef2f83-338b-4f1c-b6b3-b23e5fa85849 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:20.907613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef2f83-338b-4f1c-b6b3-b23e5fa85849 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:20.908104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef2f83-338b-4f1c-b6b3-b23e5fa85849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:20.945925 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-3aef2f83-338b-4f1c-b6b3-b23e5fa85849 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:20.947525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-3aef2f83-338b-4f1c-b6b3-b23e5fa85849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:20.995393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 71/151] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:21.001850 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849e48b-4b09-4d96-91fb-6ee07f445a6d None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:21.002405 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849e48b-4b09-4d96-91fb-6ee07f445a6d None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:21.002850 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849e48b-4b09-4d96-91fb-6ee07f445a6d None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:21.004750 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 72/152] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:21.007461 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849e48b-4b09-4d96-91fb-6ee07f445a6d None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:21.089636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 81/153] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:21.091818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42132b36-8d17-4ec1-a74d-bb7aa285366d None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:21.092106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42132b36-8d17-4ec1-a74d-bb7aa285366d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:21.092382 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42132b36-8d17-4ec1-a74d-bb7aa285366d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:21.112219 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-42132b36-8d17-4ec1-a74d-bb7aa285366d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:21.114281 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-42132b36-8d17-4ec1-a74d-bb7aa285366d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:21.136473 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849e48b-4b09-4d96-91fb-6ee07f445a6d None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:21.171961 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce0a37-7b12-4348-b227-b0809e72fddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:21.172911 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce0a37-7b12-4348-b227-b0809e72fddc None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:21.173192 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce0a37-7b12-4348-b227-b0809e72fddc None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:21.173475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce0a37-7b12-4348-b227-b0809e72fddc None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:21.174711 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce0a37-7b12-4348-b227-b0809e72fddc None admin] RBAC: Authorizing `identity:get_project(project_id=9ec214e8e5724ce6b35fe31337cb0380)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:21.176256 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce0a37-7b12-4348-b227-b0809e72fddc None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:21.178124 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 73/154] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:21 2022] GET /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:21.179307 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ceddd-3f8e-4029-af9a-5293750ed199 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:21.179640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ceddd-3f8e-4029-af9a-5293750ed199 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:21.179916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ceddd-3f8e-4029-af9a-5293750ed199 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:21.181362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 74/155] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:21.181966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ceddd-3f8e-4029-af9a-5293750ed199 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:21.197037 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 82/156] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:21.198536 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf5a1e-2403-44eb-b57c-2b33d572ffdb None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:21.198900 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf5a1e-2403-44eb-b57c-2b33d572ffdb None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:21.199178 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf5a1e-2403-44eb-b57c-2b33d572ffdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:21.230956 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-7ccf5a1e-2403-44eb-b57c-2b33d572ffdb None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:21.232741 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-7ccf5a1e-2403-44eb-b57c-2b33d572ffdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:21.303777 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 83/157] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:21.304553 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf5a1e-2403-44eb-b57c-2b33d572ffdb None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:21.330009 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ceddd-3f8e-4029-af9a-5293750ed199 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:21.363871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02863f4e-8a78-4d16-85f0-04b423591339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:21.364693 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02863f4e-8a78-4d16-85f0-04b423591339 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:21.365001 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02863f4e-8a78-4d16-85f0-04b423591339 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:21.365277 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02863f4e-8a78-4d16-85f0-04b423591339 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:21.375731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02863f4e-8a78-4d16-85f0-04b423591339 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:21.383583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-02863f4e-8a78-4d16-85f0-04b423591339 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:33:21.383583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:21.383583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:21.383583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:21.383583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:21.383583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:21.383583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:21.383583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:21.383583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:21.383583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:21.383583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:21.383583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:21.383583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:21.383583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:21.383583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:21.383583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:21.384424 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:21.384424 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:21.384424 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:21.384424 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:21.384424 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:21.384424 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:21.384424 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:21.384424 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:21.384424 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:21.384424 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:21.384424 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:21.384424 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:33:21.384424 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:21.384424 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:21.384424 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:21.384424 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:21.386440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:21.386440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:21.386440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:21.386440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:21.386440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:21.386440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:21.386440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:21.386440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:21.386440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:21.386440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:21.386440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:33:21.386440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:33:21.386440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 75/158] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:33:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:21.386440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02863f4e-8a78-4d16-85f0-04b423591339 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:21.424519 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf5a1e-2403-44eb-b57c-2b33d572ffdb None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:21.461070 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746114e0-79a3-44b2-91d6-7f2805895458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:21.461723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746114e0-79a3-44b2-91d6-7f2805895458 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:21.462001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746114e0-79a3-44b2-91d6-7f2805895458 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:21.462445 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746114e0-79a3-44b2-91d6-7f2805895458 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:21.463438 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746114e0-79a3-44b2-91d6-7f2805895458 None admin] RBAC: Adding query filter params (role.id=fec647282f5a42ab8490975708efad14, scope.project.id=9ec214e8e5724ce6b35fe31337cb0380, user.id=7683dcda9e5649fc8b526a0067af2d8b) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:21.463891 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746114e0-79a3-44b2-91d6-7f2805895458 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:21.465667 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746114e0-79a3-44b2-91d6-7f2805895458 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:21.472558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 84/159] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:33:21 2022] GET /identity/v3/role_assignments?user.id=7683dcda9e5649fc8b526a0067af2d8b&scope.project.id=9ec214e8e5724ce6b35fe31337cb0380&role.id=fec647282f5a42ab8490975708efad14 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:21.473699 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746114e0-79a3-44b2-91d6-7f2805895458 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:21.512988 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02863f4e-8a78-4d16-85f0-04b423591339 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:21.578461 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a8970-c97e-428f-9dd5-d51009b36e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:21.579279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a8970-c97e-428f-9dd5-d51009b36e0c None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:21.579573 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a8970-c97e-428f-9dd5-d51009b36e0c None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:21.579847 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a8970-c97e-428f-9dd5-d51009b36e0c None admin] PATH_INFO: `/v3/domains/default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:21.581080 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a8970-c97e-428f-9dd5-d51009b36e0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:21.582535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a8970-c97e-428f-9dd5-d51009b36e0c None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:21.583635 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746114e0-79a3-44b2-91d6-7f2805895458 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:21.584331 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 76/160] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:33:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:21.593633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a8970-c97e-428f-9dd5-d51009b36e0c None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:21.630073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca9e89-c959-48c1-b833-b1137ad2baba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:21.631261 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca9e89-c959-48c1-b833-b1137ad2baba None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:21.631755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca9e89-c959-48c1-b833-b1137ad2baba None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:21.632257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca9e89-c959-48c1-b833-b1137ad2baba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:21.651908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca9e89-c959-48c1-b833-b1137ad2baba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:21.661412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-cdca9e89-c959-48c1-b833-b1137ad2baba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:21.661412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:21.661412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:21.661412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:21.661412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:21.661412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:21.661412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:21.661412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:21.661412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:21.661412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:21.661412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:21.661412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:21.661412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:21.661412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:21.661412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:21.661412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:21.662581 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:21.662581 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:21.662581 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:21.662581 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:21.662581 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:21.662581 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:21.662581 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:21.662581 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:21.662581 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:21.662581 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:21.662581 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:21.662581 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:21.662581 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:21.662581 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:21.662581 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:21.662581 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:21.663445 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:21.663445 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:21.663445 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:21.663445 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:21.663445 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:21.663445 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:21.663445 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:21.663445 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:21.663445 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:21.663445 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:21.663445 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:21.663445 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:21.666533 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 85/161] 10.222.0.26 () {56 vars in 1114 bytes} [Mon May 9 09:33:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:21.667920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca9e89-c959-48c1-b833-b1137ad2baba None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:21.727643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a8970-c97e-428f-9dd5-d51009b36e0c None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:21.764188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70193a12-00ef-47c1-ad7f-99cf7d29d4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:21.765452 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70193a12-00ef-47c1-ad7f-99cf7d29d4c6 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:21.765452 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70193a12-00ef-47c1-ad7f-99cf7d29d4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:21.765761 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70193a12-00ef-47c1-ad7f-99cf7d29d4c6 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:21.766514 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70193a12-00ef-47c1-ad7f-99cf7d29d4c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:21.768187 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70193a12-00ef-47c1-ad7f-99cf7d29d4c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:21.768187 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70193a12-00ef-47c1-ad7f-99cf7d29d4c6 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:21.779844 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 77/162] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:21.781330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163184a7-43b0-41b5-98e6-e3326eeb4009 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:21.781560 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163184a7-43b0-41b5-98e6-e3326eeb4009 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:21.781830 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163184a7-43b0-41b5-98e6-e3326eeb4009 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:21.783206 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 78/163] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:21.783735 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163184a7-43b0-41b5-98e6-e3326eeb4009 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:21.787598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca9e89-c959-48c1-b833-b1137ad2baba None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:21.831470 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17c974-d5f7-4205-b3a4-2e4795c26beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:21.832049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17c974-d5f7-4205-b3a4-2e4795c26beb None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:21.832326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17c974-d5f7-4205-b3a4-2e4795c26beb None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:21.832595 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17c974-d5f7-4205-b3a4-2e4795c26beb None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:21.833345 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17c974-d5f7-4205-b3a4-2e4795c26beb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:21.834815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17c974-d5f7-4205-b3a4-2e4795c26beb None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:21.858857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 86/164] 10.222.0.26 () {60 vars in 1132 bytes} [Mon May 9 09:33:21 2022] POST /identity/v3/users => generated 283 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:33:21.864928 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17c974-d5f7-4205-b3a4-2e4795c26beb None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:21.906955 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163184a7-43b0-41b5-98e6-e3326eeb4009 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:21.943458 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce32245-2a2f-435b-962c-038b6f2353dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:21.944129 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce32245-2a2f-435b-962c-038b6f2353dc None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:21.944452 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce32245-2a2f-435b-962c-038b6f2353dc None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:21.944763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce32245-2a2f-435b-962c-038b6f2353dc None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:21.945517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce32245-2a2f-435b-962c-038b6f2353dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:21.945821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce32245-2a2f-435b-962c-038b6f2353dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:21.947266 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce32245-2a2f-435b-962c-038b6f2353dc None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:21.956218 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 79/165] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:33:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:21.957651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60134583-52ae-4b78-a7d0-fcc2dccf87d8 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:21.957936 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60134583-52ae-4b78-a7d0-fcc2dccf87d8 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:21.958203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60134583-52ae-4b78-a7d0-fcc2dccf87d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:21.980604 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-60134583-52ae-4b78-a7d0-fcc2dccf87d8 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:21.981827 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-60134583-52ae-4b78-a7d0-fcc2dccf87d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:21.997372 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17c974-d5f7-4205-b3a4-2e4795c26beb None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:22.033088 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45c6f7-5739-454d-b687-300bf8c9ad5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:22.034033 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45c6f7-5739-454d-b687-300bf8c9ad5e None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:22.034311 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45c6f7-5739-454d-b687-300bf8c9ad5e None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:22.034590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45c6f7-5739-454d-b687-300bf8c9ad5e None admin] PATH_INFO: `/v3/users/68d9ab86ee454d7d9d1e4d3a48048d52` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:22.035886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45c6f7-5739-454d-b687-300bf8c9ad5e None admin] RBAC: Authorizing `identity:get_user(user_id=68d9ab86ee454d7d9d1e4d3a48048d52)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:22.037391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45c6f7-5739-454d-b687-300bf8c9ad5e None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:22.039316 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 87/166] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:21 2022] GET /identity/v3/users/68d9ab86ee454d7d9d1e4d3a48048d52 => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:22.040007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45c6f7-5739-454d-b687-300bf8c9ad5e None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:22.051896 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 80/167] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:22.051896 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60134583-52ae-4b78-a7d0-fcc2dccf87d8 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:22.145983 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45c6f7-5739-454d-b687-300bf8c9ad5e None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:22.174472 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60134583-52ae-4b78-a7d0-fcc2dccf87d8 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:22.188281 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a6cb3-d65d-45a2-a465-da20871840fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:22.189251 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a6cb3-d65d-45a2-a465-da20871840fa None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:22.189529 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a6cb3-d65d-45a2-a465-da20871840fa None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:22.189813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a6cb3-d65d-45a2-a465-da20871840fa None admin] PATH_INFO: `/v3/users/7683dcda9e5649fc8b526a0067af2d8b` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:22.191024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a6cb3-d65d-45a2-a465-da20871840fa None admin] RBAC: Authorizing `identity:get_user(user_id=7683dcda9e5649fc8b526a0067af2d8b)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:22.192503 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a6cb3-d65d-45a2-a465-da20871840fa None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:22.194372 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 88/168] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:22 2022] GET /identity/v3/users/7683dcda9e5649fc8b526a0067af2d8b => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:22.195057 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a6cb3-d65d-45a2-a465-da20871840fa None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:22.222137 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b6938-fe75-46ba-a79e-ea993fa4ff97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:22.223055 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b6938-fe75-46ba-a79e-ea993fa4ff97 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:22.223506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b6938-fe75-46ba-a79e-ea993fa4ff97 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:22.223905 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b6938-fe75-46ba-a79e-ea993fa4ff97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:22.309041 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b6938-fe75-46ba-a79e-ea993fa4ff97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:22.310759 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b6938-fe75-46ba-a79e-ea993fa4ff97 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:22.325222 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a6cb3-d65d-45a2-a465-da20871840fa None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:22.377082 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1507989-5465-4795-80ae-93d2efb947ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:22.377858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1507989-5465-4795-80ae-93d2efb947ef None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:22.378137 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1507989-5465-4795-80ae-93d2efb947ef None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:22.378405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1507989-5465-4795-80ae-93d2efb947ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:22.394896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1507989-5465-4795-80ae-93d2efb947ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:22.405347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-f1507989-5465-4795-80ae-93d2efb947ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:22.405347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:22.405347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:22.405347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:22.405347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:22.405347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:22.405347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:22.405347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:22.405347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:22.405347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:22.405347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:22.405347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:22.405347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:22.405347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:22.405347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:22.405347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:22.406245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:22.406245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:22.406245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:22.406245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:22.406245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:22.406245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:22.406245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:22.406245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:22.406245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:22.406245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:22.406245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:22.406245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:22.406245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:22.406245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:22.406245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:22.406245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:22.407019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:22.407019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:22.407019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:22.407019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:22.407019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:22.407019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:22.407019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:22.407019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:22.407019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:22.407019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:22.407019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:22.407019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:22.409954 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 89/169] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:22.411105 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1507989-5465-4795-80ae-93d2efb947ef None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:22.442785 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 81/170] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:22.443275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b6938-fe75-46ba-a79e-ea993fa4ff97 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:22.543463 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1507989-5465-4795-80ae-93d2efb947ef None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:22.584981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b6938-fe75-46ba-a79e-ea993fa4ff97 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:22.597384 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f0fc7-a7f2-438e-9279-41132f58a4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:22.598843 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f0fc7-a7f2-438e-9279-41132f58a4b4 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:22.599318 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f0fc7-a7f2-438e-9279-41132f58a4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:22.600071 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f0fc7-a7f2-438e-9279-41132f58a4b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:22.626897 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c34edb-a219-4503-83e1-3618745f541a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:22.627550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c34edb-a219-4503-83e1-3618745f541a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:22.627884 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c34edb-a219-4503-83e1-3618745f541a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:22.628162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c34edb-a219-4503-83e1-3618745f541a None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:22.629008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c34edb-a219-4503-83e1-3618745f541a None admin] RBAC: Adding query filter params (name=member) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:22.629351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c34edb-a219-4503-83e1-3618745f541a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:22.630819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c34edb-a219-4503-83e1-3618745f541a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:22.640103 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 82/171] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:22.643526 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c34edb-a219-4503-83e1-3618745f541a None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:22.695608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f0fc7-a7f2-438e-9279-41132f58a4b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:22.697187 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f0fc7-a7f2-438e-9279-41132f58a4b4 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:22.754040 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c34edb-a219-4503-83e1-3618745f541a None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:22.788386 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7cc3b2-8d07-428d-84ba-b965af2ecd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:22.790175 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7cc3b2-8d07-428d-84ba-b965af2ecd16 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:22.790649 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7cc3b2-8d07-428d-84ba-b965af2ecd16 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:22.791231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7cc3b2-8d07-428d-84ba-b965af2ecd16 None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:22.793324 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7cc3b2-8d07-428d-84ba-b965af2ecd16 None admin] RBAC: Authorizing `identity:get_project(project_id=9ec214e8e5724ce6b35fe31337cb0380)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:22.794985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7cc3b2-8d07-428d-84ba-b965af2ecd16 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:22.795521 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 90/172] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:22.796432 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f0fc7-a7f2-438e-9279-41132f58a4b4 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:22.797836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 83/173] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:22 2022] GET /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:22.805606 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7cc3b2-8d07-428d-84ba-b965af2ecd16 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:22.893064 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f0fc7-a7f2-438e-9279-41132f58a4b4 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:22.926539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4dfaac-315e-436e-8b76-05dd1f53cac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:22.927412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4dfaac-315e-436e-8b76-05dd1f53cac7 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:22.927732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4dfaac-315e-436e-8b76-05dd1f53cac7 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:22.928007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4dfaac-315e-436e-8b76-05dd1f53cac7 None admin] PATH_INFO: `/v3/users/7683dcda9e5649fc8b526a0067af2d8b` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:22.929228 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4dfaac-315e-436e-8b76-05dd1f53cac7 None admin] RBAC: Authorizing `identity:get_user(user_id=7683dcda9e5649fc8b526a0067af2d8b)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:22.930681 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4dfaac-315e-436e-8b76-05dd1f53cac7 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:22.932645 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 91/174] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:22 2022] GET /identity/v3/users/7683dcda9e5649fc8b526a0067af2d8b => generated 275 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:22.940627 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4dfaac-315e-436e-8b76-05dd1f53cac7 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:22.948026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7cc3b2-8d07-428d-84ba-b965af2ecd16 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:22.997233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a608a1-a4f0-4c9b-bf40-67fe10c71ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:22.997899 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a608a1-a4f0-4c9b-bf40-67fe10c71ac4 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:22.999316 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a608a1-a4f0-4c9b-bf40-67fe10c71ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:22.999600 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a608a1-a4f0-4c9b-bf40-67fe10c71ac4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:23.000443 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a608a1-a4f0-4c9b-bf40-67fe10c71ac4 None admin] RBAC: Adding query filter params (role.id=70ac549382b14b2cbb8b35e5dfc648d4, scope.project.id=9ec214e8e5724ce6b35fe31337cb0380, user.id=68d9ab86ee454d7d9d1e4d3a48048d52) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:23.002768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a608a1-a4f0-4c9b-bf40-67fe10c71ac4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:23.004231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a608a1-a4f0-4c9b-bf40-67fe10c71ac4 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:23.016344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 84/175] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:33:22 2022] GET /identity/v3/role_assignments?user.id=68d9ab86ee454d7d9d1e4d3a48048d52&scope.project.id=9ec214e8e5724ce6b35fe31337cb0380&role.id=70ac549382b14b2cbb8b35e5dfc648d4 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:23.017182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a608a1-a4f0-4c9b-bf40-67fe10c71ac4 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:23.144061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a608a1-a4f0-4c9b-bf40-67fe10c71ac4 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:23.164945 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4dfaac-315e-436e-8b76-05dd1f53cac7 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:23.178868 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f16305-be4d-4927-8cc7-52ca338c2caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:23.179518 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f16305-be4d-4927-8cc7-52ca338c2caa None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:23.179848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f16305-be4d-4927-8cc7-52ca338c2caa None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:23.180116 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f16305-be4d-4927-8cc7-52ca338c2caa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:23.251186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f16305-be4d-4927-8cc7-52ca338c2caa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:23.252687 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f16305-be4d-4927-8cc7-52ca338c2caa None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:23.257962 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8954071-cdb7-416c-892e-ea72ecf72fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:23.258855 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8954071-cdb7-416c-892e-ea72ecf72fd8 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:23.259134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8954071-cdb7-416c-892e-ea72ecf72fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:23.259406 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8954071-cdb7-416c-892e-ea72ecf72fd8 None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:23.265551 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8954071-cdb7-416c-892e-ea72ecf72fd8 None admin] RBAC: Authorizing `identity:get_project(project_id=9ec214e8e5724ce6b35fe31337cb0380)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:23.267065 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8954071-cdb7-416c-892e-ea72ecf72fd8 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:23.268913 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 92/176] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:22 2022] GET /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380 => generated 298 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:23.274666 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8954071-cdb7-416c-892e-ea72ecf72fd8 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:23.357753 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 85/177] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:23.363237 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f16305-be4d-4927-8cc7-52ca338c2caa None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:23.473864 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f16305-be4d-4927-8cc7-52ca338c2caa None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:23.503486 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8954071-cdb7-416c-892e-ea72ecf72fd8 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:23.507410 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a39a8-b5d3-4545-8372-983fb9994b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:23.508323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a39a8-b5d3-4545-8372-983fb9994b88 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:23.508543 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a39a8-b5d3-4545-8372-983fb9994b88 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:23.508854 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a39a8-b5d3-4545-8372-983fb9994b88 None admin] PATH_INFO: `/v3/projects/c749c5092ab543298cf4557399edce98` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:23.510239 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a39a8-b5d3-4545-8372-983fb9994b88 None admin] RBAC: Authorizing `identity:get_project(project_id=c749c5092ab543298cf4557399edce98)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:23.511550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a39a8-b5d3-4545-8372-983fb9994b88 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:23.513456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 86/178] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:23 2022] GET /identity/v3/projects/c749c5092ab543298cf4557399edce98 => generated 312 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:23.514485 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4158b-6f02-46b9-a1e1-55d267f5e3d2 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:23.514760 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4158b-6f02-46b9-a1e1-55d267f5e3d2 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:23.515027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4158b-6f02-46b9-a1e1-55d267f5e3d2 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:23.521104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 87/179] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:23.521976 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4158b-6f02-46b9-a1e1-55d267f5e3d2 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:23.579212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ae89e-707d-4444-b02f-fe6ac8b1d786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:23.579880 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ae89e-707d-4444-b02f-fe6ac8b1d786 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:23.580162 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ae89e-707d-4444-b02f-fe6ac8b1d786 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:23.580436 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ae89e-707d-4444-b02f-fe6ac8b1d786 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:23.581313 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ae89e-707d-4444-b02f-fe6ac8b1d786 None admin] RBAC: Adding query filter params (role.id=0e3ae1b3fdce41c4a42a415efad3b84d, scope.project.id=9ec214e8e5724ce6b35fe31337cb0380, user.id=7683dcda9e5649fc8b526a0067af2d8b) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:23.581625 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ae89e-707d-4444-b02f-fe6ac8b1d786 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:23.583078 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ae89e-707d-4444-b02f-fe6ac8b1d786 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:23.593645 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 93/180] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:33:23 2022] GET /identity/v3/role_assignments?user.id=7683dcda9e5649fc8b526a0067af2d8b&scope.project.id=9ec214e8e5724ce6b35fe31337cb0380&role.id=0e3ae1b3fdce41c4a42a415efad3b84d => generated 608 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:23.594985 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5d3d0-6a73-47b1-b783-c36cbcd6ed6e None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:23.595274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5d3d0-6a73-47b1-b783-c36cbcd6ed6e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:23.595553 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5d3d0-6a73-47b1-b783-c36cbcd6ed6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:23.624347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-dff5d3d0-6a73-47b1-b783-c36cbcd6ed6e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:23.625581 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-dff5d3d0-6a73-47b1-b783-c36cbcd6ed6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:23.630995 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4158b-6f02-46b9-a1e1-55d267f5e3d2 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:23.667377 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df52daf-69da-4998-af3c-a956d47e9e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:23.668054 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df52daf-69da-4998-af3c-a956d47e9e18 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:23.668331 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df52daf-69da-4998-af3c-a956d47e9e18 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:23.668601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df52daf-69da-4998-af3c-a956d47e9e18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:23.669463 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df52daf-69da-4998-af3c-a956d47e9e18 None admin] RBAC: Adding query filter params (role.id=fec647282f5a42ab8490975708efad14, scope.project.id=c749c5092ab543298cf4557399edce98, user.id=7683dcda9e5649fc8b526a0067af2d8b) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:23.669769 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df52daf-69da-4998-af3c-a956d47e9e18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:23.671380 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df52daf-69da-4998-af3c-a956d47e9e18 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:23.678118 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 88/181] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:33:23 2022] GET /identity/v3/role_assignments?user.id=7683dcda9e5649fc8b526a0067af2d8b&scope.project.id=c749c5092ab543298cf4557399edce98&role.id=fec647282f5a42ab8490975708efad14 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:23.731452 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 94/182] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:23.770800 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df52daf-69da-4998-af3c-a956d47e9e18 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:23.914778 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df52daf-69da-4998-af3c-a956d47e9e18 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:23.953394 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7e350-15cb-4e6a-8585-08aaa10db004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:23.954633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7e350-15cb-4e6a-8585-08aaa10db004 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:23.955107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7e350-15cb-4e6a-8585-08aaa10db004 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:23.955568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7e350-15cb-4e6a-8585-08aaa10db004 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:23.969409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a7e350-15cb-4e6a-8585-08aaa10db004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:23.977659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-a8a7e350-15cb-4e6a-8585-08aaa10db004 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 09:33:23.977659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:23.977659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:23.977659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:23.977659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:23.977659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:23.977659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:23.977659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:23.977659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:23.977659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:23.977659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:23.977659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:23.977659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:23.977659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:23.977659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:23.977659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:23.978621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:23.978621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:23.978621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:23.978621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:23.978621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:23.978621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:23.978621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:23.978621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:23.978621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:23.978621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:23.978621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:23.978621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:33:23.978621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:23.978621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:23.978621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:23.978621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:23.979402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:23.979402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:23.979402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:23.979402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:23.979402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:23.979402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:23.979402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:23.979402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:23.979402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:23.979402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:23.979402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 09:33:23.979402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:33:23.981982 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 89/183] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:23.988113 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5d3d0-6a73-47b1-b783-c36cbcd6ed6e None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:24.106292 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5d3d0-6a73-47b1-b783-c36cbcd6ed6e None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:24.144383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21460282-e956-438a-b951-37a7c552a3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:24.145086 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21460282-e956-438a-b951-37a7c552a3e6 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:24.145359 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21460282-e956-438a-b951-37a7c552a3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:24.145628 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21460282-e956-438a-b951-37a7c552a3e6 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:24.146372 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21460282-e956-438a-b951-37a7c552a3e6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:24.146671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21460282-e956-438a-b951-37a7c552a3e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:24.148052 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21460282-e956-438a-b951-37a7c552a3e6 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:24.157105 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 95/184] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:24.162926 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7e350-15cb-4e6a-8585-08aaa10db004 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:24.268496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7e350-15cb-4e6a-8585-08aaa10db004 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:24.302099 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858b395-394a-4a67-8d83-b28b714b8832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:24.303246 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858b395-394a-4a67-8d83-b28b714b8832 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:24.303659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858b395-394a-4a67-8d83-b28b714b8832 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:24.304063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858b395-394a-4a67-8d83-b28b714b8832 None admin] PATH_INFO: `/v3/users/5943d8d8fa4848bfb1ff5fc5be6b6388` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:24.305419 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858b395-394a-4a67-8d83-b28b714b8832 None admin] RBAC: Authorizing `identity:get_user(user_id=5943d8d8fa4848bfb1ff5fc5be6b6388)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:24.307000 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858b395-394a-4a67-8d83-b28b714b8832 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:24.309210 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 90/185] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:24 2022] GET /identity/v3/users/5943d8d8fa4848bfb1ff5fc5be6b6388 => generated 289 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:24.314526 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21460282-e956-438a-b951-37a7c552a3e6 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:24.421687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21460282-e956-438a-b951-37a7c552a3e6 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:24.460064 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a280a-5758-4b19-ba59-c85edd3e3078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:24.460745 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a280a-5758-4b19-ba59-c85edd3e3078 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:24.461020 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a280a-5758-4b19-ba59-c85edd3e3078 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:24.461306 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a280a-5758-4b19-ba59-c85edd3e3078 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:24.531101 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a280a-5758-4b19-ba59-c85edd3e3078 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:24.532832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a280a-5758-4b19-ba59-c85edd3e3078 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:24.627385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 96/186] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:24.633295 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858b395-394a-4a67-8d83-b28b714b8832 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:24.735862 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858b395-394a-4a67-8d83-b28b714b8832 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:24.768213 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da5a56-c25a-4a16-82d4-133c1f27da74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:24.769097 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da5a56-c25a-4a16-82d4-133c1f27da74 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:24.769369 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da5a56-c25a-4a16-82d4-133c1f27da74 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:24.769638 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da5a56-c25a-4a16-82d4-133c1f27da74 None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:24.770819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da5a56-c25a-4a16-82d4-133c1f27da74 None admin] RBAC: Authorizing `identity:get_project(project_id=9ec214e8e5724ce6b35fe31337cb0380)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:24.772276 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da5a56-c25a-4a16-82d4-133c1f27da74 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:24.774177 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 91/187] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:24 2022] GET /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:24.779915 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a280a-5758-4b19-ba59-c85edd3e3078 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:24.879874 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a280a-5758-4b19-ba59-c85edd3e3078 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:24.915534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e5bca-2b86-42b2-9b5f-07e32d224d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:24.916192 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e5bca-2b86-42b2-9b5f-07e32d224d8c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:24.916473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e5bca-2b86-42b2-9b5f-07e32d224d8c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:24.916770 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e5bca-2b86-42b2-9b5f-07e32d224d8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:24.917679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e5bca-2b86-42b2-9b5f-07e32d224d8c None admin] RBAC: Adding query filter params (role.id=78ec5dc2a67d445389f9a7a4458a12cb, scope.project.id=9ec214e8e5724ce6b35fe31337cb0380, user.id=5943d8d8fa4848bfb1ff5fc5be6b6388) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:24.917985 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e5bca-2b86-42b2-9b5f-07e32d224d8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:24.919443 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e5bca-2b86-42b2-9b5f-07e32d224d8c None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:24.925412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 97/188] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:33:24 2022] GET /identity/v3/role_assignments?user.id=5943d8d8fa4848bfb1ff5fc5be6b6388&scope.project.id=9ec214e8e5724ce6b35fe31337cb0380&role.id=78ec5dc2a67d445389f9a7a4458a12cb => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:25.067748 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f80288-83fe-439c-88f4-5397fcceb23e None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:25.068263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f80288-83fe-439c-88f4-5397fcceb23e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:25.068652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f80288-83fe-439c-88f4-5397fcceb23e None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:25.070246 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 92/189] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:25.076330 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8da3b-cf17-4afe-86cc-33289a738d3e None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:25.076779 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8da3b-cf17-4afe-86cc-33289a738d3e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:25.077151 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8da3b-cf17-4afe-86cc-33289a738d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:25.094896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-e4a8da3b-cf17-4afe-86cc-33289a738d3e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:25.096076 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e4a8da3b-cf17-4afe-86cc-33289a738d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:25.162463 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 98/190] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:25.185474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f80288-83fe-439c-88f4-5397fcceb23e None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:25.290229 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f80288-83fe-439c-88f4-5397fcceb23e None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:25.327179 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9226f7-7255-4054-b9cf-dec282e35713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:25.327942 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9226f7-7255-4054-b9cf-dec282e35713 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:25.328226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9226f7-7255-4054-b9cf-dec282e35713 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:25.328497 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9226f7-7255-4054-b9cf-dec282e35713 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:25.339399 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9226f7-7255-4054-b9cf-dec282e35713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:25.347279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-ae9226f7-7255-4054-b9cf-dec282e35713 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:33:25.347279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:25.347279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:25.347279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:25.347279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:25.347279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:25.347279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:25.347279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:25.347279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:25.347279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:25.347279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:25.347279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:25.347279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:25.347279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:25.347279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:25.347279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:25.348173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:25.348173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:25.348173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:25.348173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:25.348173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:25.348173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:25.348173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:25.348173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:25.348173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:25.348173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:25.348173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:25.348173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:33:25.348173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:25.348173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:25.348173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:25.348173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:25.348970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:25.348970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:25.348970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:25.348970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:25.348970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:25.348970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:25.348970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:25.348970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:25.348970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:25.348970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:25.348970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:33:25.348970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:33:25.348970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 93/191] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:33:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:25.354500 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8da3b-cf17-4afe-86cc-33289a738d3e None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:25.406727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e56ca9-eb4c-4b6c-ad1c-48c3373fefde None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:25.407013 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e56ca9-eb4c-4b6c-ad1c-48c3373fefde None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:25.407278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e56ca9-eb4c-4b6c-ad1c-48c3373fefde None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:25.408677 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 94/192] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:25.413586 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ce8a9-f8a2-4728-b34e-aadc72f43f98 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:25.413868 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ce8a9-f8a2-4728-b34e-aadc72f43f98 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:25.414138 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ce8a9-f8a2-4728-b34e-aadc72f43f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:25.430990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-d38ce8a9-f8a2-4728-b34e-aadc72f43f98 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:25.432167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-d38ce8a9-f8a2-4728-b34e-aadc72f43f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:25.452038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8da3b-cf17-4afe-86cc-33289a738d3e None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:25.484941 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d478b-b2a3-491f-b501-9d9f604654de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:25.485608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d478b-b2a3-491f-b501-9d9f604654de None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:25.485890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d478b-b2a3-491f-b501-9d9f604654de None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:25.486159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d478b-b2a3-491f-b501-9d9f604654de None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:25.486909 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d478b-b2a3-491f-b501-9d9f604654de None admin] RBAC: Adding query filter params (name=admin) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:25.487209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d478b-b2a3-491f-b501-9d9f604654de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:25.488593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d478b-b2a3-491f-b501-9d9f604654de None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:25.501416 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 99/193] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:25.501416 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a24e8-f7e3-453b-997c-68a03294dc23 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:25.501416 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a24e8-f7e3-453b-997c-68a03294dc23 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:25.501416 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a24e8-f7e3-453b-997c-68a03294dc23 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:25.501416 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 100/194] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:25.502167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 95/195] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:25.502896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a24e8-f7e3-453b-997c-68a03294dc23 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:25.505092 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fede3ca-714a-4bdc-bb00-f229bedcf408 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:25.505377 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fede3ca-714a-4bdc-bb00-f229bedcf408 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:25.505789 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fede3ca-714a-4bdc-bb00-f229bedcf408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:25.521095 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-8fede3ca-714a-4bdc-bb00-f229bedcf408 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:25.522266 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-8fede3ca-714a-4bdc-bb00-f229bedcf408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:25.587450 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 96/196] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:25.589393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fede3ca-714a-4bdc-bb00-f229bedcf408 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:25.597547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a24e8-f7e3-453b-997c-68a03294dc23 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:25.630809 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb774f9f-503c-4892-8d9b-7beb2f6e60bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:25.631685 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb774f9f-503c-4892-8d9b-7beb2f6e60bc None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:25.632071 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb774f9f-503c-4892-8d9b-7beb2f6e60bc None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:25.632502 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb774f9f-503c-4892-8d9b-7beb2f6e60bc None admin] PATH_INFO: `/v3/users/cd2de645701a43329f6436acb8617ff6` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:25.645799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb774f9f-503c-4892-8d9b-7beb2f6e60bc None admin] RBAC: Authorizing `identity:get_user(user_id=cd2de645701a43329f6436acb8617ff6)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:25.647279 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb774f9f-503c-4892-8d9b-7beb2f6e60bc None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:25.649004 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 101/197] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:25 2022] GET /identity/v3/users/cd2de645701a43329f6436acb8617ff6 => generated 283 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:25.651004 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb774f9f-503c-4892-8d9b-7beb2f6e60bc None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:25.686684 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fede3ca-714a-4bdc-bb00-f229bedcf408 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:25.719859 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827cfb22-cc2f-4e04-a88f-091ee63ceba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:25.720650 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827cfb22-cc2f-4e04-a88f-091ee63ceba4 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:25.720963 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827cfb22-cc2f-4e04-a88f-091ee63ceba4 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:25.721232 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827cfb22-cc2f-4e04-a88f-091ee63ceba4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:25.731861 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827cfb22-cc2f-4e04-a88f-091ee63ceba4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:25.739903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-827cfb22-cc2f-4e04-a88f-091ee63ceba4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:25.739903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:25.739903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:25.739903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:25.739903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:25.739903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:25.739903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:25.739903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:25.739903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:25.739903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:25.739903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:25.739903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:25.739903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:25.739903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:25.739903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:25.739903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:25.740836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:25.740836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:25.740836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:25.740836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:25.740836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:25.740836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:25.740836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:25.740836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:25.740836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:25.740836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:25.740836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:25.740836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:33:25.740836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:25.740836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:25.740836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:25.740836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:25.741619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:25.741619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:25.741619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:25.741619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:25.741619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:25.741619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:25.741619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:25.741619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:25.741619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:25.741619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:25.741619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:25.741619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:33:25.741619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 97/198] 10.222.0.26 () {56 vars in 1114 bytes} [Mon May 9 09:33:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:25.742307 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827cfb22-cc2f-4e04-a88f-091ee63ceba4 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:25.752459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb774f9f-503c-4892-8d9b-7beb2f6e60bc None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:25.783599 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b456ff-5fcd-4166-8ee0-877b980df45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:25.784371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b456ff-5fcd-4166-8ee0-877b980df45e None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:25.784640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b456ff-5fcd-4166-8ee0-877b980df45e None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:25.784934 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b456ff-5fcd-4166-8ee0-877b980df45e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:25.786111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b456ff-5fcd-4166-8ee0-877b980df45e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:25.787580 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b456ff-5fcd-4166-8ee0-877b980df45e None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:25.789217 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 102/199] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:33:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:25.791091 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b456ff-5fcd-4166-8ee0-877b980df45e None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:25.839317 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827cfb22-cc2f-4e04-a88f-091ee63ceba4 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:25.870801 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee42c7-cf39-4ab3-92ac-e4a429c2bb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:25.871798 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee42c7-cf39-4ab3-92ac-e4a429c2bb87 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:25.872081 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee42c7-cf39-4ab3-92ac-e4a429c2bb87 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:25.872340 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee42c7-cf39-4ab3-92ac-e4a429c2bb87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:25.884541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b456ff-5fcd-4166-8ee0-877b980df45e None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:25.915552 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af64e0c-d350-4d31-ba19-ef3f53eb2370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:25.916205 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af64e0c-d350-4d31-ba19-ef3f53eb2370 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:25.916477 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af64e0c-d350-4d31-ba19-ef3f53eb2370 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:25.916764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af64e0c-d350-4d31-ba19-ef3f53eb2370 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:25.917584 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af64e0c-d350-4d31-ba19-ef3f53eb2370 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:25.917881 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af64e0c-d350-4d31-ba19-ef3f53eb2370 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:25.919244 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af64e0c-d350-4d31-ba19-ef3f53eb2370 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:25.926696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 103/200] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:33:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:25.927429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af64e0c-d350-4d31-ba19-ef3f53eb2370 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:25.936813 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee42c7-cf39-4ab3-92ac-e4a429c2bb87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:25.938262 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee42c7-cf39-4ab3-92ac-e4a429c2bb87 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:26.019112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af64e0c-d350-4d31-ba19-ef3f53eb2370 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:26.031591 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 98/201] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:26.032403 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee42c7-cf39-4ab3-92ac-e4a429c2bb87 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:26.050202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493de483-deb8-4f7e-a24d-e1195014867d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:26.050873 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493de483-deb8-4f7e-a24d-e1195014867d None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:26.051154 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493de483-deb8-4f7e-a24d-e1195014867d None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:26.051428 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493de483-deb8-4f7e-a24d-e1195014867d None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:26.052138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493de483-deb8-4f7e-a24d-e1195014867d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:26.053619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493de483-deb8-4f7e-a24d-e1195014867d None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:26.073738 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 104/202] 10.222.0.26 () {60 vars in 1132 bytes} [Mon May 9 09:33:25 2022] POST /identity/v3/users => generated 297 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:33:26.074554 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493de483-deb8-4f7e-a24d-e1195014867d None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:26.129878 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee42c7-cf39-4ab3-92ac-e4a429c2bb87 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:26.165457 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595ba86-1ecf-4cf4-9b0a-2963155da2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:26.166501 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595ba86-1ecf-4cf4-9b0a-2963155da2b8 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:26.166865 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595ba86-1ecf-4cf4-9b0a-2963155da2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:26.167227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595ba86-1ecf-4cf4-9b0a-2963155da2b8 None admin] PATH_INFO: `/v3/users/cd2de645701a43329f6436acb8617ff6` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:26.168632 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595ba86-1ecf-4cf4-9b0a-2963155da2b8 None admin] RBAC: Authorizing `identity:get_user(user_id=cd2de645701a43329f6436acb8617ff6)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:26.170328 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595ba86-1ecf-4cf4-9b0a-2963155da2b8 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:26.176610 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 99/203] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:26 2022] GET /identity/v3/users/cd2de645701a43329f6436acb8617ff6 => generated 283 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:26.178456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595ba86-1ecf-4cf4-9b0a-2963155da2b8 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:26.185760 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493de483-deb8-4f7e-a24d-e1195014867d None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:26.220800 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f531f0-9e0b-4f94-bf40-769a87be0df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:26.221974 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f531f0-9e0b-4f94-bf40-769a87be0df8 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:26.222259 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f531f0-9e0b-4f94-bf40-769a87be0df8 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:26.222529 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f531f0-9e0b-4f94-bf40-769a87be0df8 None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:26.223797 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f531f0-9e0b-4f94-bf40-769a87be0df8 None admin] RBAC: Authorizing `identity:get_project(project_id=9c3d1e7bc10449e2957923e1cbb5e09e)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:26.225342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f531f0-9e0b-4f94-bf40-769a87be0df8 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:26.232152 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 105/204] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:26 2022] GET /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:26.233242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f531f0-9e0b-4f94-bf40-769a87be0df8 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:26.282261 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595ba86-1ecf-4cf4-9b0a-2963155da2b8 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:26.317821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd66d75-14d5-4a9f-9911-1039d5a20de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:26.318468 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd66d75-14d5-4a9f-9911-1039d5a20de7 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:26.318741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd66d75-14d5-4a9f-9911-1039d5a20de7 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:26.319005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd66d75-14d5-4a9f-9911-1039d5a20de7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:26.344966 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f531f0-9e0b-4f94-bf40-769a87be0df8 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:26.381368 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067463e1-f557-44ad-bc38-35816f16726f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:26.383152 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067463e1-f557-44ad-bc38-35816f16726f None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:26.383523 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067463e1-f557-44ad-bc38-35816f16726f None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:26.383902 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067463e1-f557-44ad-bc38-35816f16726f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:26.384857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067463e1-f557-44ad-bc38-35816f16726f None admin] RBAC: Adding query filter params (role.id=70ac549382b14b2cbb8b35e5dfc648d4, scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e, user.id=cd2de645701a43329f6436acb8617ff6) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:26.385258 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067463e1-f557-44ad-bc38-35816f16726f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:26.386949 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067463e1-f557-44ad-bc38-35816f16726f None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:26.393158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 106/205] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:33:26 2022] GET /identity/v3/role_assignments?user.id=cd2de645701a43329f6436acb8617ff6&scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e&role.id=70ac549382b14b2cbb8b35e5dfc648d4 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:26.402478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd66d75-14d5-4a9f-9911-1039d5a20de7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:26.404144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd66d75-14d5-4a9f-9911-1039d5a20de7 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:26.510588 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 100/206] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 09:33:26.517838 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067463e1-f557-44ad-bc38-35816f16726f None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:26.623916 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067463e1-f557-44ad-bc38-35816f16726f None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:26.656823 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9252b-9477-4fc1-bde2-c721ae45508e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:26.657647 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9252b-9477-4fc1-bde2-c721ae45508e None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:26.657879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9252b-9477-4fc1-bde2-c721ae45508e None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:26.658155 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9252b-9477-4fc1-bde2-c721ae45508e None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:26.659571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9252b-9477-4fc1-bde2-c721ae45508e None admin] RBAC: Authorizing `identity:get_project(project_id=9c3d1e7bc10449e2957923e1cbb5e09e)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:26.661221 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9252b-9477-4fc1-bde2-c721ae45508e None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:26.668158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 107/207] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:26 2022] GET /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:26.671930 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd66d75-14d5-4a9f-9911-1039d5a20de7 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:26.775678 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd66d75-14d5-4a9f-9911-1039d5a20de7 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:26.809960 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890fc8ed-46a4-4f70-a11d-fcfaff694797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:26.810615 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fc8ed-46a4-4f70-a11d-fcfaff694797 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:26.810890 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fc8ed-46a4-4f70-a11d-fcfaff694797 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:26.811159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fc8ed-46a4-4f70-a11d-fcfaff694797 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:26.811984 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890fc8ed-46a4-4f70-a11d-fcfaff694797 None admin] RBAC: Adding query filter params (role.id=0e3ae1b3fdce41c4a42a415efad3b84d, scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e, user.id=cd2de645701a43329f6436acb8617ff6) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:26.812291 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890fc8ed-46a4-4f70-a11d-fcfaff694797 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:26.813776 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890fc8ed-46a4-4f70-a11d-fcfaff694797 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:26.819474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 101/208] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:33:26 2022] GET /identity/v3/role_assignments?user.id=cd2de645701a43329f6436acb8617ff6&scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e&role.id=0e3ae1b3fdce41c4a42a415efad3b84d => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:28.424566 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbbfe0-9b96-4edb-9ff6-337ebeed1e96 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:28.425131 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbbfe0-9b96-4edb-9ff6-337ebeed1e96 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:28.425131 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbbfe0-9b96-4edb-9ff6-337ebeed1e96 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:28.426656 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 108/209] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:28.432017 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9b57f-ca35-4d0d-ab7c-6a9e8e296e20 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:28.432418 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9b57f-ca35-4d0d-ab7c-6a9e8e296e20 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:28.432584 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9b57f-ca35-4d0d-ab7c-6a9e8e296e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:28.450003 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-e3d9b57f-ca35-4d0d-ab7c-6a9e8e296e20 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:28.451227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e3d9b57f-ca35-4d0d-ab7c-6a9e8e296e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:28.516475 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd33a3d-0a61-431d-a1e2-f9a4632a4a1b None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:28.516811 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd33a3d-0a61-431d-a1e2-f9a4632a4a1b None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:28.517083 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd33a3d-0a61-431d-a1e2-f9a4632a4a1b None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:28.518747 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 109/210] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:33:28.524088 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee222493-5e07-4ad5-9ae0-17b0e91d226d None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:28.524373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee222493-5e07-4ad5-9ae0-17b0e91d226d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:28.524652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee222493-5e07-4ad5-9ae0-17b0e91d226d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:28.533265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 102/211] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:33:28.542839 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-ee222493-5e07-4ad5-9ae0-17b0e91d226d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:28.544127 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-ee222493-5e07-4ad5-9ae0-17b0e91d226d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:28.557288 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9b57f-ca35-4d0d-ab7c-6a9e8e296e20 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:28.625605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 110/212] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:28.649954 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee222493-5e07-4ad5-9ae0-17b0e91d226d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:28.672319 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9b57f-ca35-4d0d-ab7c-6a9e8e296e20 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:28.704374 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92cbf30-4130-475a-9259-a1088c649e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:28.705119 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92cbf30-4130-475a-9259-a1088c649e7e None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:28.705398 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92cbf30-4130-475a-9259-a1088c649e7e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:28.705661 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92cbf30-4130-475a-9259-a1088c649e7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:28.715575 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92cbf30-4130-475a-9259-a1088c649e7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:28.723123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-c92cbf30-4130-475a-9259-a1088c649e7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:28.723123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:28.723123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:28.723123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:28.723123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:28.723123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:28.723123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:28.723123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:28.723123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:28.723123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:28.723123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:28.723123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:28.723123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:28.723123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:28.723123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:28.723123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:28.723996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:28.723996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:28.723996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:28.723996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:28.723996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:28.723996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:28.723996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:28.723996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:28.723996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:28.723996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:28.723996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:28.723996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:33:28.723996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:28.723996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:28.723996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:28.723996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:28.724836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:28.724836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:28.724836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:28.724836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:28.724836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:28.724836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:28.724836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:28.724836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:28.724836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:28.724836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:28.724836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:28.724836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:33:28.724836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 103/213] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:28.726012 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88967e6-e23c-4089-993d-f38f6910d5ee None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:28.726012 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88967e6-e23c-4089-993d-f38f6910d5ee None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:28.726012 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88967e6-e23c-4089-993d-f38f6910d5ee None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:28.727425 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 104/214] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:28.730486 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88967e6-e23c-4089-993d-f38f6910d5ee None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:28.752099 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee222493-5e07-4ad5-9ae0-17b0e91d226d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:28.786964 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b0360-88ad-4deb-8e76-c382ee920593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:28.788089 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b0360-88ad-4deb-8e76-c382ee920593 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:28.788421 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b0360-88ad-4deb-8e76-c382ee920593 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:28.788724 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b0360-88ad-4deb-8e76-c382ee920593 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:28.789916 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b0360-88ad-4deb-8e76-c382ee920593 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:28.791378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b0360-88ad-4deb-8e76-c382ee920593 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:28.797559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 111/215] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:33:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:28.797559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222608f8-9bff-4ec2-a4d9-8aae0d8dc12e None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:28.797559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222608f8-9bff-4ec2-a4d9-8aae0d8dc12e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:28.797559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222608f8-9bff-4ec2-a4d9-8aae0d8dc12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:28.812707 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-222608f8-9bff-4ec2-a4d9-8aae0d8dc12e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:28.813882 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-222608f8-9bff-4ec2-a4d9-8aae0d8dc12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:28.840880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88967e6-e23c-4089-993d-f38f6910d5ee None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:28.875689 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef845484-d4dc-447e-a6bb-ec3ef84a809b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:28.876359 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef845484-d4dc-447e-a6bb-ec3ef84a809b None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:28.876637 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef845484-d4dc-447e-a6bb-ec3ef84a809b None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:28.876929 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef845484-d4dc-447e-a6bb-ec3ef84a809b None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:28.877689 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef845484-d4dc-447e-a6bb-ec3ef84a809b None admin] RBAC: Adding query filter params (name=member) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:28.877994 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef845484-d4dc-447e-a6bb-ec3ef84a809b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:28.879385 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef845484-d4dc-447e-a6bb-ec3ef84a809b None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:28.884736 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 112/216] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:28.885778 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222608f8-9bff-4ec2-a4d9-8aae0d8dc12e None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:28.888709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 105/217] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:28.893456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef845484-d4dc-447e-a6bb-ec3ef84a809b None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:28.990815 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef845484-d4dc-447e-a6bb-ec3ef84a809b None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:29.002680 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222608f8-9bff-4ec2-a4d9-8aae0d8dc12e None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:29.025431 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bddf1e-9fc5-42fb-bf31-69e84c96cced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:29.026303 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bddf1e-9fc5-42fb-bf31-69e84c96cced None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:29.026626 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bddf1e-9fc5-42fb-bf31-69e84c96cced None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:29.026894 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bddf1e-9fc5-42fb-bf31-69e84c96cced None admin] PATH_INFO: `/v3/users/c2573a34e9934446a6b0e41035f9f0d6` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:29.035242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556c344-1721-4339-8e39-5cf247e4a426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:29.035920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556c344-1721-4339-8e39-5cf247e4a426 None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:29.036190 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556c344-1721-4339-8e39-5cf247e4a426 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:29.036455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556c344-1721-4339-8e39-5cf247e4a426 None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:29.037186 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556c344-1721-4339-8e39-5cf247e4a426 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:29.038624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556c344-1721-4339-8e39-5cf247e4a426 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:29.039694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bddf1e-9fc5-42fb-bf31-69e84c96cced None admin] RBAC: Authorizing `identity:get_user(user_id=c2573a34e9934446a6b0e41035f9f0d6)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:29.041250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bddf1e-9fc5-42fb-bf31-69e84c96cced None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:29.043085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 106/218] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:28 2022] GET /identity/v3/users/c2573a34e9934446a6b0e41035f9f0d6 => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:29.043838 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bddf1e-9fc5-42fb-bf31-69e84c96cced None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:29.059378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 113/219] 10.222.0.26 () {60 vars in 1132 bytes} [Mon May 9 09:33:28 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:33:29.066305 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae94f65-b145-45b5-870a-0dd30cff8c47 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:29.067108 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae94f65-b145-45b5-870a-0dd30cff8c47 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:29.067384 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae94f65-b145-45b5-870a-0dd30cff8c47 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:29.072717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 114/220] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:29.072717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae94f65-b145-45b5-870a-0dd30cff8c47 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:29.148241 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bddf1e-9fc5-42fb-bf31-69e84c96cced None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:29.180957 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae94f65-b145-45b5-870a-0dd30cff8c47 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:29.182919 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f46c8-db90-4c6a-9593-3604d8fa3ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:29.183716 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f46c8-db90-4c6a-9593-3604d8fa3ef9 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:29.183998 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f46c8-db90-4c6a-9593-3604d8fa3ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:29.184296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f46c8-db90-4c6a-9593-3604d8fa3ef9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:29.195547 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f46c8-db90-4c6a-9593-3604d8fa3ef9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:29.203596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-c19f46c8-db90-4c6a-9593-3604d8fa3ef9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:33:29.203596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:29.203596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:29.203596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:29.203596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:29.203596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:29.203596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:29.203596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:29.203596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:29.203596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:29.203596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:29.203596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:29.203596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:29.203596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:29.203596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:29.203596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:29.204580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:29.204580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:29.204580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:29.204580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:29.204580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:29.204580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:29.204580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:29.204580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:29.204580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:29.204580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:29.204580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:29.204580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:33:29.204580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:29.204580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:29.204580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:29.204580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:29.205417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:29.205417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:29.205417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:29.205417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:29.205417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:29.205417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:29.205417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:29.205417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:29.205417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:29.205417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:29.205417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:33:29.205417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:33:29.205417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 107/221] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:33:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:29.206107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2601b-ee5b-49f7-978a-ca06b657bb07 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:29.206388 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2601b-ee5b-49f7-978a-ca06b657bb07 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:29.206657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2601b-ee5b-49f7-978a-ca06b657bb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:29.221315 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb145bd-9697-46c1-a79d-d956c56686f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:29.221968 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb145bd-9697-46c1-a79d-d956c56686f1 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:29.222244 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb145bd-9697-46c1-a79d-d956c56686f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:29.222512 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb145bd-9697-46c1-a79d-d956c56686f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:29.223786 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-7dd2601b-ee5b-49f7-978a-ca06b657bb07 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:29.225016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-7dd2601b-ee5b-49f7-978a-ca06b657bb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:29.291118 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb145bd-9697-46c1-a79d-d956c56686f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:29.291378 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 108/222] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:29.292401 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2601b-ee5b-49f7-978a-ca06b657bb07 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:29.292654 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb145bd-9697-46c1-a79d-d956c56686f1 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:29.407605 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2601b-ee5b-49f7-978a-ca06b657bb07 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:29.414593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 115/223] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:29.416169 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb145bd-9697-46c1-a79d-d956c56686f1 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:29.447492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce830cc-449f-4cc9-94f1-8d971d68f8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:29.448181 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce830cc-449f-4cc9-94f1-8d971d68f8bf None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:29.448462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce830cc-449f-4cc9-94f1-8d971d68f8bf None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:29.448786 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce830cc-449f-4cc9-94f1-8d971d68f8bf None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:29.449672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce830cc-449f-4cc9-94f1-8d971d68f8bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:29.449867 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce830cc-449f-4cc9-94f1-8d971d68f8bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:29.451220 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce830cc-449f-4cc9-94f1-8d971d68f8bf None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:29.459725 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 109/224] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:29.460540 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce830cc-449f-4cc9-94f1-8d971d68f8bf None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:29.517462 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb145bd-9697-46c1-a79d-d956c56686f1 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:29.551253 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e0e55-748a-412a-8a62-bc3d6f2076c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:29.552427 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e0e55-748a-412a-8a62-bc3d6f2076c6 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:29.552916 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e0e55-748a-412a-8a62-bc3d6f2076c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:29.553374 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e0e55-748a-412a-8a62-bc3d6f2076c6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:29.557108 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce830cc-449f-4cc9-94f1-8d971d68f8bf None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:29.564361 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e0e55-748a-412a-8a62-bc3d6f2076c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:29.572623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-965e0e55-748a-412a-8a62-bc3d6f2076c6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:29.572623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:29.572623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:29.572623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:29.572623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:29.572623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:29.572623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:29.572623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:29.572623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:29.572623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:29.572623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:29.572623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:29.572623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:29.572623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:29.572623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:29.572623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:29.573813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:29.573813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:29.573813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:29.573813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:29.573813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:29.573813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:29.573813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:29.573813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:29.573813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:29.573813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:29.573813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:29.573813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:29.573813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:29.573813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:29.573813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:29.573813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:29.574600 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:29.574600 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:29.574600 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:29.574600 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:29.574600 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:29.574600 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:29.574600 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:29.574600 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:29.574600 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:29.574600 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:29.574600 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:29.574600 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:29.575214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 116/225] 10.222.0.26 () {56 vars in 1114 bytes} [Mon May 9 09:33:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:29.580223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e0e55-748a-412a-8a62-bc3d6f2076c6 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:29.590525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0c539-1226-4d55-93e8-c418aa93e9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:29.591395 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0c539-1226-4d55-93e8-c418aa93e9e1 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:29.591670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0c539-1226-4d55-93e8-c418aa93e9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:29.591939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0c539-1226-4d55-93e8-c418aa93e9e1 None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:29.593378 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0c539-1226-4d55-93e8-c418aa93e9e1 None admin] RBAC: Authorizing `identity:get_project(project_id=9c3d1e7bc10449e2957923e1cbb5e09e)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:29.594832 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0c539-1226-4d55-93e8-c418aa93e9e1 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:29.596838 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 110/226] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:29 2022] GET /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e => generated 302 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:29.597601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0c539-1226-4d55-93e8-c418aa93e9e1 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:29.681473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e0e55-748a-412a-8a62-bc3d6f2076c6 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:29.694701 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0c539-1226-4d55-93e8-c418aa93e9e1 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:29.714454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ecfb3c-4c7a-4cd6-8675-e028a1342c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:29.715065 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecfb3c-4c7a-4cd6-8675-e028a1342c11 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:29.715403 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecfb3c-4c7a-4cd6-8675-e028a1342c11 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:29.715699 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecfb3c-4c7a-4cd6-8675-e028a1342c11 None admin] PATH_INFO: `/v3/users/cd2de645701a43329f6436acb8617ff6` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:29.716809 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ecfb3c-4c7a-4cd6-8675-e028a1342c11 None admin] RBAC: Authorizing `identity:get_user(user_id=cd2de645701a43329f6436acb8617ff6)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:29.718260 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ecfb3c-4c7a-4cd6-8675-e028a1342c11 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:29.720119 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 117/227] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:29 2022] GET /identity/v3/users/cd2de645701a43329f6436acb8617ff6 => generated 283 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:29.721006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ecfb3c-4c7a-4cd6-8675-e028a1342c11 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:29.736808 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b773b-4442-4a3d-97bc-cba6effc97d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:29.737851 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b773b-4442-4a3d-97bc-cba6effc97d3 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:29.738337 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b773b-4442-4a3d-97bc-cba6effc97d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:29.738821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b773b-4442-4a3d-97bc-cba6effc97d3 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:29.739767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b773b-4442-4a3d-97bc-cba6effc97d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:29.740276 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b773b-4442-4a3d-97bc-cba6effc97d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:29.741894 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b773b-4442-4a3d-97bc-cba6effc97d3 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:29.750995 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 111/228] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:33:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:29.752092 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b773b-4442-4a3d-97bc-cba6effc97d3 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:29.842797 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ecfb3c-4c7a-4cd6-8675-e028a1342c11 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:29.855757 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b773b-4442-4a3d-97bc-cba6effc97d3 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:29.885186 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77366181-98ca-40d3-bea2-b1084f7a3e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:29.885862 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77366181-98ca-40d3-bea2-b1084f7a3e92 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:29.886293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77366181-98ca-40d3-bea2-b1084f7a3e92 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:29.886567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77366181-98ca-40d3-bea2-b1084f7a3e92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:29.887767 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77366181-98ca-40d3-bea2-b1084f7a3e92 None admin] RBAC: Adding query filter params (role.id=fec647282f5a42ab8490975708efad14, scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e, user.id=c2573a34e9934446a6b0e41035f9f0d6) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:29.888239 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77366181-98ca-40d3-bea2-b1084f7a3e92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:29.889806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77366181-98ca-40d3-bea2-b1084f7a3e92 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:29.892272 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ac66c-2cb1-4d93-ac02-d002bef14697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:29.893396 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ac66c-2cb1-4d93-ac02-d002bef14697 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:29.893905 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ac66c-2cb1-4d93-ac02-d002bef14697 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:29.894370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ac66c-2cb1-4d93-ac02-d002bef14697 None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:29.895438 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 118/229] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:33:29 2022] GET /identity/v3/role_assignments?user.id=c2573a34e9934446a6b0e41035f9f0d6&scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e&role.id=fec647282f5a42ab8490975708efad14 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:29.896051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77366181-98ca-40d3-bea2-b1084f7a3e92 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:29.896620 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ac66c-2cb1-4d93-ac02-d002bef14697 None admin] RBAC: Authorizing `identity:get_project(project_id=9c3d1e7bc10449e2957923e1cbb5e09e)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:29.898329 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ac66c-2cb1-4d93-ac02-d002bef14697 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:29.905613 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 112/230] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:29 2022] GET /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:29.906913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ac66c-2cb1-4d93-ac02-d002bef14697 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:30.018803 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ac66c-2cb1-4d93-ac02-d002bef14697 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:30.031994 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77366181-98ca-40d3-bea2-b1084f7a3e92 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:30.070871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2d2fb-62b1-462c-9261-0c5dbfd31489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:30.072135 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2d2fb-62b1-462c-9261-0c5dbfd31489 None admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:30.072619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2d2fb-62b1-462c-9261-0c5dbfd31489 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:30.073112 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2d2fb-62b1-462c-9261-0c5dbfd31489 None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e/users/cd2de645701a43329f6436acb8617ff6/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:30.075651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2d2fb-62b1-462c-9261-0c5dbfd31489 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c3d1e7bc10449e2957923e1cbb5e09e, user_id=cd2de645701a43329f6436acb8617ff6, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:30.077672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2d2fb-62b1-462c-9261-0c5dbfd31489 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:30.079910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91377987-20fb-4c19-a1e3-3b6012c29941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:30.082391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91377987-20fb-4c19-a1e3-3b6012c29941 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:30.082391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91377987-20fb-4c19-a1e3-3b6012c29941 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:30.082391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91377987-20fb-4c19-a1e3-3b6012c29941 None admin] PATH_INFO: `/v3/users/cd2de645701a43329f6436acb8617ff6` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:30.084752 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91377987-20fb-4c19-a1e3-3b6012c29941 None admin] RBAC: Authorizing `identity:get_user(user_id=cd2de645701a43329f6436acb8617ff6)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:30.087474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91377987-20fb-4c19-a1e3-3b6012c29941 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:30.089890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 119/231] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:29 2022] GET /identity/v3/users/cd2de645701a43329f6436acb8617ff6 => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:30.095949 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91377987-20fb-4c19-a1e3-3b6012c29941 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:30.115883 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 113/232] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:33:29 2022] PUT /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e/users/cd2de645701a43329f6436acb8617ff6/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:33:30.212148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91377987-20fb-4c19-a1e3-3b6012c29941 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:30.250838 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca1cf1-f8f3-48da-983c-15b163990074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:30.251973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca1cf1-f8f3-48da-983c-15b163990074 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:30.252403 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca1cf1-f8f3-48da-983c-15b163990074 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:30.252880 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca1cf1-f8f3-48da-983c-15b163990074 None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:30.254315 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca1cf1-f8f3-48da-983c-15b163990074 None admin] RBAC: Authorizing `identity:get_project(project_id=9c3d1e7bc10449e2957923e1cbb5e09e)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:30.256042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca1cf1-f8f3-48da-983c-15b163990074 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:30.258059 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 120/233] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:30 2022] GET /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:30.264032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2d2fb-62b1-462c-9261-0c5dbfd31489 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:30.389592 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2d2fb-62b1-462c-9261-0c5dbfd31489 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:30.427000 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da77e85-1858-4fd3-b8a6-effc4643a5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:30.428083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da77e85-1858-4fd3-b8a6-effc4643a5f0 None admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:30.428370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da77e85-1858-4fd3-b8a6-effc4643a5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:30.428639 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da77e85-1858-4fd3-b8a6-effc4643a5f0 None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e/users/cd2de645701a43329f6436acb8617ff6/roles/0e3ae1b3fdce41c4a42a415efad3b84d` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:30.430878 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da77e85-1858-4fd3-b8a6-effc4643a5f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c3d1e7bc10449e2957923e1cbb5e09e, user_id=cd2de645701a43329f6436acb8617ff6, role_id=0e3ae1b3fdce41c4a42a415efad3b84d)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:30.432356 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da77e85-1858-4fd3-b8a6-effc4643a5f0 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:30.450486 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 114/234] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:33:30 2022] PUT /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e/users/cd2de645701a43329f6436acb8617ff6/roles/0e3ae1b3fdce41c4a42a415efad3b84d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:33:31.769625 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e96d1-cc91-4cda-bcf3-216e65d22ff3 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:31.770648 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e96d1-cc91-4cda-bcf3-216e65d22ff3 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:31.771125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e96d1-cc91-4cda-bcf3-216e65d22ff3 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:31.772957 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 121/235] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:31.778685 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c596775-6f70-44f1-bc0d-02232eabb346 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:31.779546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c596775-6f70-44f1-bc0d-02232eabb346 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:31.780021 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c596775-6f70-44f1-bc0d-02232eabb346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:31.802323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-0c596775-6f70-44f1-bc0d-02232eabb346 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:31.803804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-0c596775-6f70-44f1-bc0d-02232eabb346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:31.814522 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc7df4-d6b3-4ceb-b17b-c45b0337a277 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:31.815097 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc7df4-d6b3-4ceb-b17b-c45b0337a277 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:31.815568 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc7df4-d6b3-4ceb-b17b-c45b0337a277 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:31.817415 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 122/236] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:31.822871 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cb75f-a6ba-48a2-93d9-85df363e58e3 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:31.823152 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cb75f-a6ba-48a2-93d9-85df363e58e3 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:31.823493 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cb75f-a6ba-48a2-93d9-85df363e58e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:31.856079 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-c11cb75f-a6ba-48a2-93d9-85df363e58e3 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:31.857275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c11cb75f-a6ba-48a2-93d9-85df363e58e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:31.903365 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 115/237] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:31.955916 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 123/238] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:31.957266 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c596775-6f70-44f1-bc0d-02232eabb346 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:31.988688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11cb75f-a6ba-48a2-93d9-85df363e58e3 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:32.060741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c596775-6f70-44f1-bc0d-02232eabb346 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:32.106706 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d698866-c059-4f5e-8f41-463c63be6189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:32.107443 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d698866-c059-4f5e-8f41-463c63be6189 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:32.107715 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d698866-c059-4f5e-8f41-463c63be6189 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:32.107984 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d698866-c059-4f5e-8f41-463c63be6189 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:32.110251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d698866-c059-4f5e-8f41-463c63be6189 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:32.111727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d698866-c059-4f5e-8f41-463c63be6189 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:32.113400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 116/239] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:33:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:32.123418 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d698866-c059-4f5e-8f41-463c63be6189 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:32.164084 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11cb75f-a6ba-48a2-93d9-85df363e58e3 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:32.197440 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a368c-dea1-443f-9ca2-c43d6c1328b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:32.198246 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a368c-dea1-443f-9ca2-c43d6c1328b2 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:32.198567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a368c-dea1-443f-9ca2-c43d6c1328b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:32.198840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a368c-dea1-443f-9ca2-c43d6c1328b2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:32.209091 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a368c-dea1-443f-9ca2-c43d6c1328b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:32.217125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-601a368c-dea1-443f-9ca2-c43d6c1328b2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 09:33:32.217125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:32.217125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:32.217125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:32.217125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:32.217125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:32.217125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:32.217125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:32.217125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:32.217125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:32.217125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:32.217125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:32.217125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:32.217125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:32.217125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:32.217125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:32.218029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:32.218029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:32.218029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:32.218029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:32.218029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:32.218029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:32.218029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:32.218029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:32.218029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:32.218029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:32.218029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:32.218029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:32.218029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:32.218029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:32.218029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:32.218029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:32.218814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:32.218814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:32.218814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:32.218814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:32.218814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:32.218814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:32.218814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:32.218814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:32.218814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:32.218814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:32.218814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 09:33:32.218814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:32.218814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 124/240] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:32.224057 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a368c-dea1-443f-9ca2-c43d6c1328b2 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:32.318020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d698866-c059-4f5e-8f41-463c63be6189 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:32.380865 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a500d8a-7af2-46ff-922f-cc60ed91334d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:32.381463 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a500d8a-7af2-46ff-922f-cc60ed91334d None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:32.381744 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a500d8a-7af2-46ff-922f-cc60ed91334d None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:32.382017 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a500d8a-7af2-46ff-922f-cc60ed91334d None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:32.382747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a500d8a-7af2-46ff-922f-cc60ed91334d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:32.384202 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a500d8a-7af2-46ff-922f-cc60ed91334d None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:32.397745 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a368c-dea1-443f-9ca2-c43d6c1328b2 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:32.431153 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 117/241] 10.222.0.26 () {60 vars in 1132 bytes} [Mon May 9 09:33:32 2022] POST /identity/v3/users => generated 293 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:33:32.443318 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f06fcb-eebc-47d7-9e38-a06b7434d464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:32.444056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f06fcb-eebc-47d7-9e38-a06b7434d464 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:32.444331 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f06fcb-eebc-47d7-9e38-a06b7434d464 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:32.444601 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f06fcb-eebc-47d7-9e38-a06b7434d464 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:32.445396 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f06fcb-eebc-47d7-9e38-a06b7434d464 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:32.445746 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f06fcb-eebc-47d7-9e38-a06b7434d464 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:32.447339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f06fcb-eebc-47d7-9e38-a06b7434d464 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:32.459489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 125/242] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:32.467331 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a500d8a-7af2-46ff-922f-cc60ed91334d None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:32.574740 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a500d8a-7af2-46ff-922f-cc60ed91334d None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:32.610439 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead5d9a-eb31-414b-8534-493996c75978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:32.611328 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead5d9a-eb31-414b-8534-493996c75978 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:32.611599 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead5d9a-eb31-414b-8534-493996c75978 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:32.611865 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead5d9a-eb31-414b-8534-493996c75978 None admin] PATH_INFO: `/v3/users/2c9c56bff0664840ac81860a642bc08b` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:32.627083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead5d9a-eb31-414b-8534-493996c75978 None admin] RBAC: Authorizing `identity:get_user(user_id=2c9c56bff0664840ac81860a642bc08b)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:32.628578 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead5d9a-eb31-414b-8534-493996c75978 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:32.630742 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 118/243] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:32 2022] GET /identity/v3/users/2c9c56bff0664840ac81860a642bc08b => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:32.637905 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f06fcb-eebc-47d7-9e38-a06b7434d464 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:32.788232 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f06fcb-eebc-47d7-9e38-a06b7434d464 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:32.829361 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc273ac-5f2d-4774-a6f8-ebfdb19422de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:32.830284 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc273ac-5f2d-4774-a6f8-ebfdb19422de None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:32.830703 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc273ac-5f2d-4774-a6f8-ebfdb19422de None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:32.831116 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc273ac-5f2d-4774-a6f8-ebfdb19422de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:32.850154 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56f2f9-6b97-48ce-b6c5-9538168e7fa5 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:32.850707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56f2f9-6b97-48ce-b6c5-9538168e7fa5 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:32.850898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56f2f9-6b97-48ce-b6c5-9538168e7fa5 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:32.853615 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 119/244] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:33:32.859392 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824e95d-ea2d-47fc-8734-6197fe589bb2 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:32.859777 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824e95d-ea2d-47fc-8734-6197fe589bb2 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:32.860143 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824e95d-ea2d-47fc-8734-6197fe589bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:32.877136 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-7824e95d-ea2d-47fc-8734-6197fe589bb2 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:32.878463 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-7824e95d-ea2d-47fc-8734-6197fe589bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:32.903869 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc273ac-5f2d-4774-a6f8-ebfdb19422de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:32.905605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc273ac-5f2d-4774-a6f8-ebfdb19422de None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:32.960656 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 120/245] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:33:32.966501 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bbc93-d630-4a21-bc24-c3ffe94e02c4 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:32.966936 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bbc93-d630-4a21-bc24-c3ffe94e02c4 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:32.967398 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bbc93-d630-4a21-bc24-c3ffe94e02c4 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:32.969518 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 121/246] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:32.975058 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449fc703-22a2-4f92-b026-1651741ba8b9 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:32.975408 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449fc703-22a2-4f92-b026-1651741ba8b9 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:32.975754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449fc703-22a2-4f92-b026-1651741ba8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:32.994876 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-449fc703-22a2-4f92-b026-1651741ba8b9 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:32.996704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-449fc703-22a2-4f92-b026-1651741ba8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:33.026265 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 126/247] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:33.027323 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc273ac-5f2d-4774-a6f8-ebfdb19422de None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:33.082798 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 122/248] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:33:33.084061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449fc703-22a2-4f92-b026-1651741ba8b9 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:33.146523 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc273ac-5f2d-4774-a6f8-ebfdb19422de None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:33.183119 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4615c-94f2-4c8f-9216-f4bfccf19727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:33.183968 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4615c-94f2-4c8f-9216-f4bfccf19727 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:33.184340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4615c-94f2-4c8f-9216-f4bfccf19727 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:33.184695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4615c-94f2-4c8f-9216-f4bfccf19727 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:33.189171 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449fc703-22a2-4f92-b026-1651741ba8b9 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:33.195237 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4615c-94f2-4c8f-9216-f4bfccf19727 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:33.203491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-06d4615c-94f2-4c8f-9216-f4bfccf19727 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:33.203491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:33.203491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:33.203491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:33.203491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:33.203491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:33.203491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:33.203491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:33.203491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:33.203491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:33.203491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:33.203491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:33.203491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:33.203491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:33.203491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:33.203491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:33.204405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:33.204405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:33.204405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:33.204405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:33.204405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:33.204405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:33.204405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:33.204405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:33.204405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:33.204405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:33.204405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:33.204405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:33.204405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:33.204405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:33.204405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:33.204405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:33.205257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:33.205257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:33.205257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:33.205257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:33.205257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:33.205257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:33.205257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:33.205257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:33.205257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:33.205257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:33.205257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:33.205257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:33.205257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 127/249] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:33.205935 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4615c-94f2-4c8f-9216-f4bfccf19727 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:33.233262 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303b288-5f63-4fb2-8a63-a65a65bc01b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:33.234426 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303b288-5f63-4fb2-8a63-a65a65bc01b5 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:33.234703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303b288-5f63-4fb2-8a63-a65a65bc01b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:33.234976 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303b288-5f63-4fb2-8a63-a65a65bc01b5 None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:33.236722 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c303b288-5f63-4fb2-8a63-a65a65bc01b5 None admin] RBAC: Authorizing `identity:get_project(project_id=9c3d1e7bc10449e2957923e1cbb5e09e)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:33.238673 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c303b288-5f63-4fb2-8a63-a65a65bc01b5 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:33.241568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 123/250] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:33 2022] GET /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:33.242412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303b288-5f63-4fb2-8a63-a65a65bc01b5 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:33.320646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4615c-94f2-4c8f-9216-f4bfccf19727 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:33.346055 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303b288-5f63-4fb2-8a63-a65a65bc01b5 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:33.353887 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e560f-5335-4f29-b2ca-31fd63a5fdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:33.354770 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e560f-5335-4f29-b2ca-31fd63a5fdd6 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:33.355052 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e560f-5335-4f29-b2ca-31fd63a5fdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:33.355326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e560f-5335-4f29-b2ca-31fd63a5fdd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:33.365859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e560f-5335-4f29-b2ca-31fd63a5fdd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:33.373984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-b99e560f-5335-4f29-b2ca-31fd63a5fdd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:33:33.373984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:33.373984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:33.373984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:33.373984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:33.373984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:33.373984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:33.373984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:33.373984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:33.373984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:33.373984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:33.373984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:33.373984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:33.373984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:33.373984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:33.373984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:33.376081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:33.376081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:33.376081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:33.376081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:33.376081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:33.376081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:33.376081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:33.376081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:33.376081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:33.376081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:33.376081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:33.376081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:33.376081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:33.376081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:33.376081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:33.376081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:33.378181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:33.378181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:33.378181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:33.378181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:33.378181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:33.378181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:33.378181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:33.378181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:33.378181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:33.378181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:33.378181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:33:33.378181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:33.378181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 128/251] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:33:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:33.378181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e560f-5335-4f29-b2ca-31fd63a5fdd6 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:33.388686 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc09dd0-735b-4f33-b915-baf224d38d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:33.388686 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc09dd0-735b-4f33-b915-baf224d38d85 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:33.388686 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc09dd0-735b-4f33-b915-baf224d38d85 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:33.388686 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc09dd0-735b-4f33-b915-baf224d38d85 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:33.389407 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc09dd0-735b-4f33-b915-baf224d38d85 None admin] RBAC: Adding query filter params (name=member) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:33.389407 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc09dd0-735b-4f33-b915-baf224d38d85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:33.389407 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc09dd0-735b-4f33-b915-baf224d38d85 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:33.397606 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 124/252] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:33.399125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc09dd0-735b-4f33-b915-baf224d38d85 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:33.518649 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc09dd0-735b-4f33-b915-baf224d38d85 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:33.525183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e560f-5335-4f29-b2ca-31fd63a5fdd6 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:33.551482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbb701-a5f9-40ed-afac-b747d9f0956c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:33.552146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbb701-a5f9-40ed-afac-b747d9f0956c None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:33.552416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbb701-a5f9-40ed-afac-b747d9f0956c None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:33.552704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbb701-a5f9-40ed-afac-b747d9f0956c None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:33.553453 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbb701-a5f9-40ed-afac-b747d9f0956c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:33.553748 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbb701-a5f9-40ed-afac-b747d9f0956c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:33.555148 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbb701-a5f9-40ed-afac-b747d9f0956c None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:33.567348 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 125/253] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:33.568269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbb701-a5f9-40ed-afac-b747d9f0956c None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:33.579597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ac082-68db-4c8f-a294-ef74e354b15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:33.580258 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ac082-68db-4c8f-a294-ef74e354b15e None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:33.580723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ac082-68db-4c8f-a294-ef74e354b15e None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:33.581148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ac082-68db-4c8f-a294-ef74e354b15e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:33.582048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ac082-68db-4c8f-a294-ef74e354b15e None admin] RBAC: Adding query filter params (role.id=78ec5dc2a67d445389f9a7a4458a12cb, scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e, user.id=2c9c56bff0664840ac81860a642bc08b) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:33.582598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ac082-68db-4c8f-a294-ef74e354b15e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:33.584120 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ac082-68db-4c8f-a294-ef74e354b15e None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:33.590124 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 129/254] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:33:33 2022] GET /identity/v3/role_assignments?user.id=2c9c56bff0664840ac81860a642bc08b&scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e&role.id=78ec5dc2a67d445389f9a7a4458a12cb => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:33.591570 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccce9b-43cd-4a91-b3c0-9a8df66b9ac9 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:33.592027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccce9b-43cd-4a91-b3c0-9a8df66b9ac9 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:33.592489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccce9b-43cd-4a91-b3c0-9a8df66b9ac9 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:33.593998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 130/255] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:33.594941 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccce9b-43cd-4a91-b3c0-9a8df66b9ac9 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:33.691270 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbb701-a5f9-40ed-afac-b747d9f0956c None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:33.729754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60e863-cc06-46ef-886b-2a07faa89444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:33.730935 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60e863-cc06-46ef-886b-2a07faa89444 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:33.731359 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60e863-cc06-46ef-886b-2a07faa89444 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:33.731773 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60e863-cc06-46ef-886b-2a07faa89444 None admin] PATH_INFO: `/v3/users/c2573a34e9934446a6b0e41035f9f0d6` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:33.733438 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60e863-cc06-46ef-886b-2a07faa89444 None admin] RBAC: Authorizing `identity:get_user(user_id=c2573a34e9934446a6b0e41035f9f0d6)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:33.735406 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60e863-cc06-46ef-886b-2a07faa89444 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:33.737363 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 126/256] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:33 2022] GET /identity/v3/users/c2573a34e9934446a6b0e41035f9f0d6 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:33.739661 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4f348-d848-4632-8108-5d85541868c8 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:33.740266 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4f348-d848-4632-8108-5d85541868c8 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:33.740712 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4f348-d848-4632-8108-5d85541868c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:33.761133 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-e6e4f348-d848-4632-8108-5d85541868c8 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:33.763697 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e6e4f348-d848-4632-8108-5d85541868c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:33.818525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccce9b-43cd-4a91-b3c0-9a8df66b9ac9 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:33.853161 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be4599-9b0b-44fc-ac73-6af1085fa18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:33.854041 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be4599-9b0b-44fc-ac73-6af1085fa18e None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:33.854321 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be4599-9b0b-44fc-ac73-6af1085fa18e None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:33.854643 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be4599-9b0b-44fc-ac73-6af1085fa18e None admin] PATH_INFO: `/v3/users/cd2de645701a43329f6436acb8617ff6` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:33.858025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 127/257] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:33.859479 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be4599-9b0b-44fc-ac73-6af1085fa18e None admin] RBAC: Authorizing `identity:get_user(user_id=cd2de645701a43329f6436acb8617ff6)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:33.860988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be4599-9b0b-44fc-ac73-6af1085fa18e None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:33.862988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 131/258] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:33 2022] GET /identity/v3/users/cd2de645701a43329f6436acb8617ff6 => generated 283 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:33.864946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4f348-d848-4632-8108-5d85541868c8 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:33.869677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be4599-9b0b-44fc-ac73-6af1085fa18e None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:34.040962 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be4599-9b0b-44fc-ac73-6af1085fa18e None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:34.072309 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4f348-d848-4632-8108-5d85541868c8 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:34.096539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c58d7-4e8e-47d7-b315-0418ad7fe9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:34.097224 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c58d7-4e8e-47d7-b315-0418ad7fe9cc None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:34.097497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c58d7-4e8e-47d7-b315-0418ad7fe9cc None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:34.097764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c58d7-4e8e-47d7-b315-0418ad7fe9cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:34.154698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cee29-23f0-4309-b1d9-35ca10347726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:34.155580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cee29-23f0-4309-b1d9-35ca10347726 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:34.155857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cee29-23f0-4309-b1d9-35ca10347726 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:34.156128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cee29-23f0-4309-b1d9-35ca10347726 None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:34.161412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407cee29-23f0-4309-b1d9-35ca10347726 None admin] RBAC: Authorizing `identity:get_project(project_id=9c3d1e7bc10449e2957923e1cbb5e09e)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:34.162915 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407cee29-23f0-4309-b1d9-35ca10347726 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:34.164706 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 128/259] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:33 2022] GET /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e => generated 302 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:34.165477 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cee29-23f0-4309-b1d9-35ca10347726 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:34.175109 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c58d7-4e8e-47d7-b315-0418ad7fe9cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:34.176645 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c58d7-4e8e-47d7-b315-0418ad7fe9cc None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:34.275801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 132/260] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:34.276770 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c58d7-4e8e-47d7-b315-0418ad7fe9cc None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:34.371737 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cee29-23f0-4309-b1d9-35ca10347726 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:34.374353 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c58d7-4e8e-47d7-b315-0418ad7fe9cc None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:34.410594 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc72e4-6a27-4018-90f9-d771f3695f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:34.414238 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc72e4-6a27-4018-90f9-d771f3695f7f None admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:34.414531 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc72e4-6a27-4018-90f9-d771f3695f7f None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:34.414819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc72e4-6a27-4018-90f9-d771f3695f7f None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e/users/c2573a34e9934446a6b0e41035f9f0d6/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:34.417905 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc72e4-6a27-4018-90f9-d771f3695f7f None admin] RBAC: Authorizing `identity:create_grant(project_id=9c3d1e7bc10449e2957923e1cbb5e09e, user_id=c2573a34e9934446a6b0e41035f9f0d6, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:34.419355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc72e4-6a27-4018-90f9-d771f3695f7f None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:34.440398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 133/261] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:33:34 2022] PUT /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e/users/c2573a34e9934446a6b0e41035f9f0d6/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:33:34.441246 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc72e4-6a27-4018-90f9-d771f3695f7f None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:34.442870 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67334f7f-a516-48b1-b589-63b73efda834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:34.444875 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67334f7f-a516-48b1-b589-63b73efda834 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:34.445151 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67334f7f-a516-48b1-b589-63b73efda834 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:34.445422 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67334f7f-a516-48b1-b589-63b73efda834 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:34.469356 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67334f7f-a516-48b1-b589-63b73efda834 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:34.477660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-67334f7f-a516-48b1-b589-63b73efda834 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:34.477660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:34.477660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:34.477660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:34.477660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:34.477660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:34.477660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:34.477660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:34.477660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:34.477660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:34.477660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:34.477660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:34.477660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:34.477660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:34.477660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:34.477660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:34.478627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:34.478627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:34.478627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:34.478627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:34.478627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:34.478627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:34.478627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:34.478627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:34.478627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:34.478627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:34.478627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:34.478627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:33:34.478627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:34.478627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:34.478627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:34.478627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:34.479474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:34.479474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:34.479474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:34.479474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:34.479474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:34.479474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:34.479474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:34.479474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:34.479474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:34.479474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:34.479474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:34.479474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:33:34.482085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 129/262] 10.222.0.26 () {56 vars in 1114 bytes} [Mon May 9 09:33:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:34.492607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67334f7f-a516-48b1-b589-63b73efda834 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:34.646087 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc72e4-6a27-4018-90f9-d771f3695f7f None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:34.668251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67334f7f-a516-48b1-b589-63b73efda834 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:34.706430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0facc-5d7a-497c-bc31-a35169d6b813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:34.707310 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0facc-5d7a-497c-bc31-a35169d6b813 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:34.707589 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0facc-5d7a-497c-bc31-a35169d6b813 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:34.707865 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0facc-5d7a-497c-bc31-a35169d6b813 None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:34.709211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0facc-5d7a-497c-bc31-a35169d6b813 None admin] RBAC: Authorizing `identity:get_project(project_id=9c3d1e7bc10449e2957923e1cbb5e09e)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:34.710691 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0facc-5d7a-497c-bc31-a35169d6b813 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:34.713405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 134/263] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:34 2022] GET /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:34.721863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0facc-5d7a-497c-bc31-a35169d6b813 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:34.735683 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3171775-74e7-4078-a6d3-bd915f1a4213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:34.736411 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3171775-74e7-4078-a6d3-bd915f1a4213 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:34.736711 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3171775-74e7-4078-a6d3-bd915f1a4213 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:34.736986 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3171775-74e7-4078-a6d3-bd915f1a4213 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:34.737747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3171775-74e7-4078-a6d3-bd915f1a4213 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:34.739212 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3171775-74e7-4078-a6d3-bd915f1a4213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:34.740652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3171775-74e7-4078-a6d3-bd915f1a4213 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:34.754966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 130/264] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:33:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:34.762538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3171775-74e7-4078-a6d3-bd915f1a4213 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:34.911237 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0facc-5d7a-497c-bc31-a35169d6b813 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:34.938261 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3171775-74e7-4078-a6d3-bd915f1a4213 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:34.974149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ead91d-6ca7-49ea-b544-9e5cefd0d4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:34.974806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ead91d-6ca7-49ea-b544-9e5cefd0d4cd None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:34.975086 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ead91d-6ca7-49ea-b544-9e5cefd0d4cd None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:34.975359 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ead91d-6ca7-49ea-b544-9e5cefd0d4cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:34.976195 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ead91d-6ca7-49ea-b544-9e5cefd0d4cd None admin] RBAC: Adding query filter params (role.id=70ac549382b14b2cbb8b35e5dfc648d4, scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e, user.id=cd2de645701a43329f6436acb8617ff6) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:34.976499 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ead91d-6ca7-49ea-b544-9e5cefd0d4cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:34.977995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ead91d-6ca7-49ea-b544-9e5cefd0d4cd None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:34.990066 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 135/265] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:33:34 2022] GET /identity/v3/role_assignments?user.id=cd2de645701a43329f6436acb8617ff6&scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e&role.id=70ac549382b14b2cbb8b35e5dfc648d4 => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:34.995719 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28017332-30e1-45ec-80c1-e725c465af66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:34.997208 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28017332-30e1-45ec-80c1-e725c465af66 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:34.997485 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28017332-30e1-45ec-80c1-e725c465af66 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:34.997755 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28017332-30e1-45ec-80c1-e725c465af66 None admin] PATH_INFO: `/v3/users/cd2de645701a43329f6436acb8617ff6` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:34.999057 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28017332-30e1-45ec-80c1-e725c465af66 None admin] RBAC: Authorizing `identity:get_user(user_id=cd2de645701a43329f6436acb8617ff6)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:35.000545 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28017332-30e1-45ec-80c1-e725c465af66 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:35.002306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 131/266] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:34 2022] GET /identity/v3/users/cd2de645701a43329f6436acb8617ff6 => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:35.008082 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ead91d-6ca7-49ea-b544-9e5cefd0d4cd None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:35.134663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ead91d-6ca7-49ea-b544-9e5cefd0d4cd None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:35.178989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921de00-aaf4-4d41-be86-0eb9b2230385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:35.179550 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921de00-aaf4-4d41-be86-0eb9b2230385 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:35.179831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921de00-aaf4-4d41-be86-0eb9b2230385 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:35.180099 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921de00-aaf4-4d41-be86-0eb9b2230385 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:35.267585 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5921de00-aaf4-4d41-be86-0eb9b2230385 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:35.269195 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5921de00-aaf4-4d41-be86-0eb9b2230385 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:35.379579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 136/267] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:35.385123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28017332-30e1-45ec-80c1-e725c465af66 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:35.488338 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28017332-30e1-45ec-80c1-e725c465af66 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:35.530115 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db41f9-2c6b-48c1-9cee-ea1ab8749545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:35.531016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db41f9-2c6b-48c1-9cee-ea1ab8749545 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:35.531293 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db41f9-2c6b-48c1-9cee-ea1ab8749545 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:35.531562 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db41f9-2c6b-48c1-9cee-ea1ab8749545 None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:35.532911 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db41f9-2c6b-48c1-9cee-ea1ab8749545 None admin] RBAC: Authorizing `identity:get_project(project_id=9c3d1e7bc10449e2957923e1cbb5e09e)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:35.534427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db41f9-2c6b-48c1-9cee-ea1ab8749545 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:35.536220 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 132/268] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:35 2022] GET /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:35.547588 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921de00-aaf4-4d41-be86-0eb9b2230385 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:35.665858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921de00-aaf4-4d41-be86-0eb9b2230385 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:35.706224 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ed79e-f730-49be-8230-24dcea8f69f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:35.707273 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ed79e-f730-49be-8230-24dcea8f69f4 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:35.707750 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ed79e-f730-49be-8230-24dcea8f69f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:35.708211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ed79e-f730-49be-8230-24dcea8f69f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:35.709278 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ed79e-f730-49be-8230-24dcea8f69f4 None admin] RBAC: Adding query filter params (role.id=0e3ae1b3fdce41c4a42a415efad3b84d, scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e, user.id=cd2de645701a43329f6436acb8617ff6) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:35.709786 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ed79e-f730-49be-8230-24dcea8f69f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:35.711499 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ed79e-f730-49be-8230-24dcea8f69f4 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:35.719897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 137/269] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:33:35 2022] GET /identity/v3/role_assignments?user.id=cd2de645701a43329f6436acb8617ff6&scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e&role.id=0e3ae1b3fdce41c4a42a415efad3b84d => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:35.729153 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30335cbc-b18b-4655-bfa8-82e6a8ce6790 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:35.729434 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30335cbc-b18b-4655-bfa8-82e6a8ce6790 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:35.730002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30335cbc-b18b-4655-bfa8-82e6a8ce6790 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:35.731707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 133/270] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:35.737787 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d34964-dfe4-432d-a238-6234c884df8f None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:35.738070 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d34964-dfe4-432d-a238-6234c884df8f None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:35.738334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d34964-dfe4-432d-a238-6234c884df8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:35.757266 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-06d34964-dfe4-432d-a238-6234c884df8f None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:35.758806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-06d34964-dfe4-432d-a238-6234c884df8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:35.828112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 138/271] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:35.852090 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30335cbc-b18b-4655-bfa8-82e6a8ce6790 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:35.993154 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30335cbc-b18b-4655-bfa8-82e6a8ce6790 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:36.029809 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb4319-5db5-423e-a14f-6decc0ed0f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:36.030579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb4319-5db5-423e-a14f-6decc0ed0f54 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:36.030858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb4319-5db5-423e-a14f-6decc0ed0f54 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:36.031123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb4319-5db5-423e-a14f-6decc0ed0f54 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:36.041421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb4319-5db5-423e-a14f-6decc0ed0f54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:36.049350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-b0eb4319-5db5-423e-a14f-6decc0ed0f54 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:36.049350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:36.049350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:36.049350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:36.049350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:36.049350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:36.049350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:36.049350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:36.049350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:36.049350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:36.049350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:36.049350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:36.049350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:36.049350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:36.049350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:36.049350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:36.050335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:36.050335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:36.050335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:36.050335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:36.050335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:36.050335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:36.050335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:36.050335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:36.050335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:36.050335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:36.050335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:36.050335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:33:36.050335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:36.050335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:36.050335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:36.050335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:36.051147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:36.051147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:36.051147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:36.051147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:36.051147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:36.051147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:36.051147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:36.051147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:36.051147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:36.051147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:36.051147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:36.051147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:33:36.051147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 134/272] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:36.058303 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d34964-dfe4-432d-a238-6234c884df8f None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:36.149180 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4ae64-7993-42cf-8079-848de8fe66f3 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:36.149807 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4ae64-7993-42cf-8079-848de8fe66f3 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:36.150280 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4ae64-7993-42cf-8079-848de8fe66f3 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:36.151946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 135/273] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:33:36.157456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865435cb-0a38-4873-babb-a6fba9e49786 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:36.158007 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865435cb-0a38-4873-babb-a6fba9e49786 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:36.158479 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865435cb-0a38-4873-babb-a6fba9e49786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:36.178490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-865435cb-0a38-4873-babb-a6fba9e49786 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:36.180038 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-865435cb-0a38-4873-babb-a6fba9e49786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:36.185827 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d34964-dfe4-432d-a238-6234c884df8f None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:36.219131 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b9813-5ab1-4c42-b339-2682c2872a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:36.220067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b9813-5ab1-4c42-b339-2682c2872a4d None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:36.220525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b9813-5ab1-4c42-b339-2682c2872a4d None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:36.220960 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b9813-5ab1-4c42-b339-2682c2872a4d None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:36.221878 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b9813-5ab1-4c42-b339-2682c2872a4d None admin] RBAC: Adding query filter params (name=member) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:36.222335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b9813-5ab1-4c42-b339-2682c2872a4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:36.223893 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b9813-5ab1-4c42-b339-2682c2872a4d None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:36.233420 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 139/274] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:36.240780 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b9813-5ab1-4c42-b339-2682c2872a4d None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:36.254669 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 136/275] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:33:36.278744 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865435cb-0a38-4873-babb-a6fba9e49786 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:36.351262 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b9813-5ab1-4c42-b339-2682c2872a4d None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:36.380037 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865435cb-0a38-4873-babb-a6fba9e49786 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:36.388807 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c53b7c-04b5-471e-a41c-8a21d0d19a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:36.390038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c53b7c-04b5-471e-a41c-8a21d0d19a61 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:36.390616 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c53b7c-04b5-471e-a41c-8a21d0d19a61 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:36.390897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c53b7c-04b5-471e-a41c-8a21d0d19a61 None admin] PATH_INFO: `/v3/users/7e33ee3c09ae45e98eda8324e030d4ec` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:36.404732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c53b7c-04b5-471e-a41c-8a21d0d19a61 None admin] RBAC: Authorizing `identity:get_user(user_id=7e33ee3c09ae45e98eda8324e030d4ec)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:36.406293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c53b7c-04b5-471e-a41c-8a21d0d19a61 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:36.412680 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 140/276] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:36 2022] GET /identity/v3/users/7e33ee3c09ae45e98eda8324e030d4ec => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:36.412680 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d006b9-73c6-47ce-89ee-b32932c5da0f None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:36.412680 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d006b9-73c6-47ce-89ee-b32932c5da0f None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:36.412680 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d006b9-73c6-47ce-89ee-b32932c5da0f None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:36.412680 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 141/277] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:36.415660 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d006b9-73c6-47ce-89ee-b32932c5da0f None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:36.417340 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988472df-db9f-4ca4-b24a-cdb1f2e6f6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:36.418136 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988472df-db9f-4ca4-b24a-cdb1f2e6f6f6 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:36.418406 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988472df-db9f-4ca4-b24a-cdb1f2e6f6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:36.418669 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988472df-db9f-4ca4-b24a-cdb1f2e6f6f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:36.419810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988472df-db9f-4ca4-b24a-cdb1f2e6f6f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:36.421290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988472df-db9f-4ca4-b24a-cdb1f2e6f6f6 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:36.422884 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 137/278] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:33:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:36.424110 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c6eab-bd25-4913-8317-23683f170e56 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:36.424385 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c6eab-bd25-4913-8317-23683f170e56 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:36.424651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c6eab-bd25-4913-8317-23683f170e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:36.440406 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-9b8c6eab-bd25-4913-8317-23683f170e56 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:36.441603 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-9b8c6eab-bd25-4913-8317-23683f170e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:36.504370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 138/279] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:36.505192 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c6eab-bd25-4913-8317-23683f170e56 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:36.524899 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d006b9-73c6-47ce-89ee-b32932c5da0f None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:36.567723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc1fed-2f1a-4057-b3f1-450eab75a073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:36.568458 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc1fed-2f1a-4057-b3f1-450eab75a073 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:36.568845 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc1fed-2f1a-4057-b3f1-450eab75a073 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:36.569119 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc1fed-2f1a-4057-b3f1-450eab75a073 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:36.569950 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc1fed-2f1a-4057-b3f1-450eab75a073 None admin] RBAC: Adding query filter params (role.id=fec647282f5a42ab8490975708efad14, user.id=7e33ee3c09ae45e98eda8324e030d4ec, scope.system=all) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:36.570259 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc1fed-2f1a-4057-b3f1-450eab75a073 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:36.571702 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc1fed-2f1a-4057-b3f1-450eab75a073 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:36.576656 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 142/280] 10.222.0.26 () {56 vars in 1308 bytes} [Mon May 9 09:33:36 2022] GET /identity/v3/role_assignments?user.id=7e33ee3c09ae45e98eda8324e030d4ec&scope.system=all&role.id=fec647282f5a42ab8490975708efad14 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:36.589802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc1fed-2f1a-4057-b3f1-450eab75a073 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:36.610679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c6eab-bd25-4913-8317-23683f170e56 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:36.655315 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705ecd4-ffef-49bd-bfbd-c708cce2c1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:36.656003 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705ecd4-ffef-49bd-bfbd-c708cce2c1bb None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:36.656284 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705ecd4-ffef-49bd-bfbd-c708cce2c1bb None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:36.656555 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705ecd4-ffef-49bd-bfbd-c708cce2c1bb None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:36.657309 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705ecd4-ffef-49bd-bfbd-c708cce2c1bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:36.658858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705ecd4-ffef-49bd-bfbd-c708cce2c1bb None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:36.683949 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 139/281] 10.222.0.26 () {60 vars in 1132 bytes} [Mon May 9 09:33:36 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:33:36.717506 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc1fed-2f1a-4057-b3f1-450eab75a073 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:36.758885 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f04a9e-456a-47ce-9b03-6b09afcafc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:36.759983 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f04a9e-456a-47ce-9b03-6b09afcafc3b None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:36.760403 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f04a9e-456a-47ce-9b03-6b09afcafc3b None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:36.760910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f04a9e-456a-47ce-9b03-6b09afcafc3b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:36.775867 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f04a9e-456a-47ce-9b03-6b09afcafc3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:36.787036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-33f04a9e-456a-47ce-9b03-6b09afcafc3b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 09:33:36.787036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:36.787036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:36.787036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:36.787036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:36.787036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:36.787036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:36.787036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:36.787036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:36.787036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:36.787036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:36.787036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:36.787036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:36.787036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:36.787036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:36.787036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:36.787999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:36.787999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:36.787999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:36.787999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:36.787999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:36.787999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:36.787999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:36.787999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:36.787999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:36.787999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:36.787999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:36.787999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:36.787999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:36.787999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:36.787999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:36.787999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:36.788840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:36.788840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:36.788840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:36.788840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:36.788840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:36.788840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:36.788840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:36.788840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:36.788840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:36.788840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:36.788840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 09:33:36.788840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:36.788840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 143/282] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:36.797233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705ecd4-ffef-49bd-bfbd-c708cce2c1bb None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:36.903232 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705ecd4-ffef-49bd-bfbd-c708cce2c1bb None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:36.936703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445355f1-3442-4ee3-8f79-ee9da3e41a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:36.937427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445355f1-3442-4ee3-8f79-ee9da3e41a8f None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:36.937703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445355f1-3442-4ee3-8f79-ee9da3e41a8f None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:36.937970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445355f1-3442-4ee3-8f79-ee9da3e41a8f None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:36.938720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445355f1-3442-4ee3-8f79-ee9da3e41a8f None admin] RBAC: Adding query filter params (name=reader) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:36.939067 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445355f1-3442-4ee3-8f79-ee9da3e41a8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:36.940461 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445355f1-3442-4ee3-8f79-ee9da3e41a8f None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:36.949665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 140/283] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:36.955162 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f04a9e-456a-47ce-9b03-6b09afcafc3b None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:37.058029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f04a9e-456a-47ce-9b03-6b09afcafc3b None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:37.092500 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1d035-94dd-4c07-a61b-6b948799f148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:37.093414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1d035-94dd-4c07-a61b-6b948799f148 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:37.093685 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1d035-94dd-4c07-a61b-6b948799f148 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:37.093949 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1d035-94dd-4c07-a61b-6b948799f148 None admin] PATH_INFO: `/v3/users/2c9c56bff0664840ac81860a642bc08b` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:37.095161 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1d035-94dd-4c07-a61b-6b948799f148 None admin] RBAC: Authorizing `identity:get_user(user_id=2c9c56bff0664840ac81860a642bc08b)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:37.096824 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1d035-94dd-4c07-a61b-6b948799f148 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:37.099409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 144/284] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:36 2022] GET /identity/v3/users/2c9c56bff0664840ac81860a642bc08b => generated 297 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:37.106414 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445355f1-3442-4ee3-8f79-ee9da3e41a8f None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:37.226000 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445355f1-3442-4ee3-8f79-ee9da3e41a8f None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:37.270523 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb3e08-18c5-4de2-8146-fc9fc5db6fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:37.271394 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb3e08-18c5-4de2-8146-fc9fc5db6fd0 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:37.271675 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb3e08-18c5-4de2-8146-fc9fc5db6fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:37.271951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb3e08-18c5-4de2-8146-fc9fc5db6fd0 None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:37.273355 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb3e08-18c5-4de2-8146-fc9fc5db6fd0 None admin] RBAC: Authorizing `identity:get_project(project_id=9c3d1e7bc10449e2957923e1cbb5e09e)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:37.274821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb3e08-18c5-4de2-8146-fc9fc5db6fd0 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:37.276796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 141/285] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:37 2022] GET /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:37.285786 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1d035-94dd-4c07-a61b-6b948799f148 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:37.363032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732469e7-76a4-4e1b-bbe1-7eaf43c66edb None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:37.363379 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732469e7-76a4-4e1b-bbe1-7eaf43c66edb None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:37.363587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732469e7-76a4-4e1b-bbe1-7eaf43c66edb None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:37.364994 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 142/286] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:37.370263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e9ee0-258e-4066-b217-321e2e7c377e None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:37.370554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e9ee0-258e-4066-b217-321e2e7c377e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:37.370833 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e9ee0-258e-4066-b217-321e2e7c377e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:37.390550 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1d035-94dd-4c07-a61b-6b948799f148 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:37.405987 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-f09e9ee0-258e-4066-b217-321e2e7c377e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:37.407196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f09e9ee0-258e-4066-b217-321e2e7c377e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:37.423454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ec515-c904-451c-a8fd-f66cb2c37890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:37.424343 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ec515-c904-451c-a8fd-f66cb2c37890 None admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:37.424615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ec515-c904-451c-a8fd-f66cb2c37890 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:37.424910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ec515-c904-451c-a8fd-f66cb2c37890 None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e/users/2c9c56bff0664840ac81860a642bc08b/roles/78ec5dc2a67d445389f9a7a4458a12cb` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:37.426854 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ec515-c904-451c-a8fd-f66cb2c37890 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c3d1e7bc10449e2957923e1cbb5e09e, user_id=2c9c56bff0664840ac81860a642bc08b, role_id=78ec5dc2a67d445389f9a7a4458a12cb)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:37.428314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ec515-c904-451c-a8fd-f66cb2c37890 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:37.448475 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 145/287] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:33:37 2022] PUT /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e/users/2c9c56bff0664840ac81860a642bc08b/roles/78ec5dc2a67d445389f9a7a4458a12cb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:33:37.506478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 143/288] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:33:37.537492 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ec515-c904-451c-a8fd-f66cb2c37890 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:37.642646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ec515-c904-451c-a8fd-f66cb2c37890 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:37.678343 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d5921-e301-498a-8b5b-9e1d96d4071c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:37.679159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d5921-e301-498a-8b5b-9e1d96d4071c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:37.679431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d5921-e301-498a-8b5b-9e1d96d4071c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:37.679818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d5921-e301-498a-8b5b-9e1d96d4071c None admin] PATH_INFO: `/v3/roles/member` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:37.691962 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d5921-e301-498a-8b5b-9e1d96d4071c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:37.699995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-dd6d5921-e301-498a-8b5b-9e1d96d4071c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:37.699995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:37.699995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:37.699995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:37.699995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:37.699995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:37.699995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:37.699995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:37.699995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:37.699995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:37.699995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:37.699995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:37.699995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:37.699995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:37.699995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:37.699995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:37.700987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:37.700987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:37.700987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:37.700987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:37.700987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:37.700987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:37.700987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:37.700987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:37.700987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:37.700987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:37.700987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:37.700987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:37.700987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:37.700987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:37.700987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:37.700987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:37.701803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:37.701803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:37.701803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:37.701803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:37.701803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:37.701803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:37.701803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:37.701803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:37.701803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:37.701803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:37.701803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:37.701803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:37.701803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 146/289] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:37.710397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e9ee0-258e-4066-b217-321e2e7c377e None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:37.817630 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e9ee0-258e-4066-b217-321e2e7c377e None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:37.854873 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36173dfd-e727-47d5-9d70-70e0baf83996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:37.855549 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36173dfd-e727-47d5-9d70-70e0baf83996 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:37.855826 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36173dfd-e727-47d5-9d70-70e0baf83996 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:37.856097 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36173dfd-e727-47d5-9d70-70e0baf83996 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:37.856884 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36173dfd-e727-47d5-9d70-70e0baf83996 None admin] RBAC: Adding query filter params (name=member) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:37.857184 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36173dfd-e727-47d5-9d70-70e0baf83996 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:37.858566 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36173dfd-e727-47d5-9d70-70e0baf83996 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:37.867740 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 144/290] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:37.873651 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d5921-e301-498a-8b5b-9e1d96d4071c None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:37.999766 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d5921-e301-498a-8b5b-9e1d96d4071c None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:38.032168 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4e2db-7dda-47d6-b1a3-6fc751148b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:38.033075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4e2db-7dda-47d6-b1a3-6fc751148b3e None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:38.033352 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4e2db-7dda-47d6-b1a3-6fc751148b3e None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:38.033616 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4e2db-7dda-47d6-b1a3-6fc751148b3e None admin] PATH_INFO: `/v3/users/c2573a34e9934446a6b0e41035f9f0d6` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:38.034811 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4e2db-7dda-47d6-b1a3-6fc751148b3e None admin] RBAC: Authorizing `identity:get_user(user_id=c2573a34e9934446a6b0e41035f9f0d6)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:38.036270 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4e2db-7dda-47d6-b1a3-6fc751148b3e None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:38.038383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 147/291] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:37 2022] GET /identity/v3/users/c2573a34e9934446a6b0e41035f9f0d6 => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:38.043622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36173dfd-e727-47d5-9d70-70e0baf83996 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:38.196366 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36173dfd-e727-47d5-9d70-70e0baf83996 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:38.248720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55c190-4b5d-442c-ad43-165f0990068e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:38.249809 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55c190-4b5d-442c-ad43-165f0990068e None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:38.250294 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55c190-4b5d-442c-ad43-165f0990068e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:38.250764 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55c190-4b5d-442c-ad43-165f0990068e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:38.344137 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55c190-4b5d-442c-ad43-165f0990068e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:38.345990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55c190-4b5d-442c-ad43-165f0990068e None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:38.470198 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 145/292] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:38.475687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4e2db-7dda-47d6-b1a3-6fc751148b3e None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:38.621989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4e2db-7dda-47d6-b1a3-6fc751148b3e None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:38.659586 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585bd2a-e5b1-482a-8ef3-be5b252ef32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:38.660854 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585bd2a-e5b1-482a-8ef3-be5b252ef32c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:38.661362 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585bd2a-e5b1-482a-8ef3-be5b252ef32c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:38.661765 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585bd2a-e5b1-482a-8ef3-be5b252ef32c None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:38.664558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585bd2a-e5b1-482a-8ef3-be5b252ef32c None admin] RBAC: Authorizing `identity:get_project(project_id=9c3d1e7bc10449e2957923e1cbb5e09e)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:38.666333 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585bd2a-e5b1-482a-8ef3-be5b252ef32c None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:38.668837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 148/293] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:38 2022] GET /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:38.677850 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55c190-4b5d-442c-ad43-165f0990068e None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:38.836318 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55c190-4b5d-442c-ad43-165f0990068e None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:38.883182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d4291-8bb7-4e7f-8db5-939280bac28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:38.884216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d4291-8bb7-4e7f-8db5-939280bac28f None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:38.884753 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d4291-8bb7-4e7f-8db5-939280bac28f None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:38.885221 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d4291-8bb7-4e7f-8db5-939280bac28f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:38.886301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d4291-8bb7-4e7f-8db5-939280bac28f None admin] RBAC: Adding query filter params (role.id=fec647282f5a42ab8490975708efad14, scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e, user.id=c2573a34e9934446a6b0e41035f9f0d6) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:38.886811 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d4291-8bb7-4e7f-8db5-939280bac28f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:38.888500 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d4291-8bb7-4e7f-8db5-939280bac28f None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:38.897481 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 146/294] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:33:38 2022] GET /identity/v3/role_assignments?user.id=c2573a34e9934446a6b0e41035f9f0d6&scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e&role.id=fec647282f5a42ab8490975708efad14 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:39.241586 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081005f-e9a3-4798-a55f-f6bdbd553f0d None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:39.241926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081005f-e9a3-4798-a55f-f6bdbd553f0d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:39.242059 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081005f-e9a3-4798-a55f-f6bdbd553f0d None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:39.243853 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 149/295] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:39.249327 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9223c7-1d86-405b-aaea-e9eb2a80623e None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:39.250164 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9223c7-1d86-405b-aaea-e9eb2a80623e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:39.250645 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9223c7-1d86-405b-aaea-e9eb2a80623e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:39.271071 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-1f9223c7-1d86-405b-aaea-e9eb2a80623e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:39.272595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-1f9223c7-1d86-405b-aaea-e9eb2a80623e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:39.342428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 147/296] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:39.369687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081005f-e9a3-4798-a55f-f6bdbd553f0d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:39.488007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081005f-e9a3-4798-a55f-f6bdbd553f0d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:39.526702 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8510b2-250b-433c-bb07-5ca577e1acbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:39.527886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8510b2-250b-433c-bb07-5ca577e1acbe None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:39.528394 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8510b2-250b-433c-bb07-5ca577e1acbe None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:39.528888 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8510b2-250b-433c-bb07-5ca577e1acbe None admin] PATH_INFO: `/v3/roles/member` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:39.539641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8510b2-250b-433c-bb07-5ca577e1acbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:39.550562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-9e8510b2-250b-433c-bb07-5ca577e1acbe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:39.550562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:39.550562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:39.550562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:39.550562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:39.550562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:39.550562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:39.550562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:39.550562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:39.550562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:39.550562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:39.550562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:39.550562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:39.550562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:39.550562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:39.550562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:39.551501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:39.551501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:39.551501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:39.551501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:39.551501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:39.551501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:39.551501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:39.551501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:39.551501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:39.551501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:39.551501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:39.551501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:39.551501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:39.551501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:39.551501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:39.551501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:39.552301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:39.552301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:39.552301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:39.552301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:39.552301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:39.552301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:39.552301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:39.552301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:39.552301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:39.552301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:39.552301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:39.552301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:39.555050 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 150/297] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:39.561299 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9223c7-1d86-405b-aaea-e9eb2a80623e None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:39.579114 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c18196-9072-4ab6-9d6b-7397e1bfe31c None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:39.579415 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c18196-9072-4ab6-9d6b-7397e1bfe31c None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:39.579651 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c18196-9072-4ab6-9d6b-7397e1bfe31c None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:39.581300 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 151/298] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:33:39.589373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7380ec1-9f49-4c13-a8bb-97f720b718ce None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:39.589650 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7380ec1-9f49-4c13-a8bb-97f720b718ce None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:39.589921 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7380ec1-9f49-4c13-a8bb-97f720b718ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:39.610846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-a7380ec1-9f49-4c13-a8bb-97f720b718ce None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:39.612424 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-a7380ec1-9f49-4c13-a8bb-97f720b718ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:39.668524 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9223c7-1d86-405b-aaea-e9eb2a80623e None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:39.692382 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 152/299] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:39.696706 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255219b-782f-4dfe-ba2e-17dc855d89f8 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:39.696706 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255219b-782f-4dfe-ba2e-17dc855d89f8 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:39.696706 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255219b-782f-4dfe-ba2e-17dc855d89f8 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:39.696706 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 153/300] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:39.700744 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7a404-f209-4cc3-8869-e8708da4d5e3 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:39.701023 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7a404-f209-4cc3-8869-e8708da4d5e3 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:39.701289 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7a404-f209-4cc3-8869-e8708da4d5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:39.716471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f4063-bb9c-4f22-80b8-b0bb010b8d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:39.717055 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-8bf7a404-f209-4cc3-8869-e8708da4d5e3 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:39.717845 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f4063-bb9c-4f22-80b8-b0bb010b8d27 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:39.718129 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f4063-bb9c-4f22-80b8-b0bb010b8d27 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:39.718244 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-8bf7a404-f209-4cc3-8869-e8708da4d5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:39.718403 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f4063-bb9c-4f22-80b8-b0bb010b8d27 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:39.719155 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f4063-bb9c-4f22-80b8-b0bb010b8d27 None admin] RBAC: Adding query filter params (name=member) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:39.719458 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f4063-bb9c-4f22-80b8-b0bb010b8d27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:39.720880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f4063-bb9c-4f22-80b8-b0bb010b8d27 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:39.729972 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 148/301] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:39.730985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f4063-bb9c-4f22-80b8-b0bb010b8d27 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:39.787886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 154/302] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:33:39.793588 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7a404-f209-4cc3-8869-e8708da4d5e3 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:39.833164 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f4063-bb9c-4f22-80b8-b0bb010b8d27 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:39.865220 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435891a-248d-4d6e-abdc-371d7e805e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:39.866008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435891a-248d-4d6e-abdc-371d7e805e5d None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:39.867283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435891a-248d-4d6e-abdc-371d7e805e5d None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:39.867556 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435891a-248d-4d6e-abdc-371d7e805e5d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:39.868876 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6435891a-248d-4d6e-abdc-371d7e805e5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:39.870483 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6435891a-248d-4d6e-abdc-371d7e805e5d None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:39.872439 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 149/303] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:33:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:39.875864 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435891a-248d-4d6e-abdc-371d7e805e5d None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:39.902157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7a404-f209-4cc3-8869-e8708da4d5e3 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:39.936492 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7aef6-265f-44b1-93f5-0fb2f0d0e088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:39.937396 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7aef6-265f-44b1-93f5-0fb2f0d0e088 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:39.937668 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7aef6-265f-44b1-93f5-0fb2f0d0e088 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:39.937935 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7aef6-265f-44b1-93f5-0fb2f0d0e088 None admin] PATH_INFO: `/v3/users/7e33ee3c09ae45e98eda8324e030d4ec` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:39.941293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7aef6-265f-44b1-93f5-0fb2f0d0e088 None admin] RBAC: Authorizing `identity:get_user(user_id=7e33ee3c09ae45e98eda8324e030d4ec)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:39.942740 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7aef6-265f-44b1-93f5-0fb2f0d0e088 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:39.948902 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 155/304] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:39 2022] GET /identity/v3/users/7e33ee3c09ae45e98eda8324e030d4ec => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:39.948902 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7aef6-265f-44b1-93f5-0fb2f0d0e088 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:39.981833 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435891a-248d-4d6e-abdc-371d7e805e5d None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:40.017040 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84397b01-3968-487d-a8c1-c7cf0ee731e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:40.017834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84397b01-3968-487d-a8c1-c7cf0ee731e5 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:40.018107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84397b01-3968-487d-a8c1-c7cf0ee731e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:40.018370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84397b01-3968-487d-a8c1-c7cf0ee731e5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:40.029422 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84397b01-3968-487d-a8c1-c7cf0ee731e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:40.037618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-84397b01-3968-487d-a8c1-c7cf0ee731e5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 09:33:40.037618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:40.037618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:40.037618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:40.037618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:40.037618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:40.037618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:40.037618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:40.037618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:40.037618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:40.037618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:40.037618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:40.037618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:40.037618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:40.037618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:40.037618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:40.038648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:40.038648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:40.038648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:40.038648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:40.038648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:40.038648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:40.038648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:40.038648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:40.038648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:40.038648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:40.038648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:40.038648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:33:40.038648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:40.038648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:40.038648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:40.038648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:40.039456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:40.039456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:40.039456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:40.039456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:40.039456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:40.039456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:40.039456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:40.039456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:40.039456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:40.039456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:40.039456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 09:33:40.039456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:33:40.039456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 150/305] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:40.044975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84397b01-3968-487d-a8c1-c7cf0ee731e5 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:40.045965 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7aef6-265f-44b1-93f5-0fb2f0d0e088 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:40.079842 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f905e27-1aa8-4cb5-8772-06c9fd5ca69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:40.080869 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f905e27-1aa8-4cb5-8772-06c9fd5ca69d None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:40.081146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f905e27-1aa8-4cb5-8772-06c9fd5ca69d None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:40.081412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f905e27-1aa8-4cb5-8772-06c9fd5ca69d None admin] PATH_INFO: `/v3/groups` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:40.082101 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f905e27-1aa8-4cb5-8772-06c9fd5ca69d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:40.083533 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f905e27-1aa8-4cb5-8772-06c9fd5ca69d None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:40.108691 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 156/306] 10.222.0.26 () {60 vars in 1134 bytes} [Mon May 9 09:33:39 2022] POST /identity/v3/groups => generated 229 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:33:40.108691 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f905e27-1aa8-4cb5-8772-06c9fd5ca69d None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:40.164189 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84397b01-3968-487d-a8c1-c7cf0ee731e5 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:40.198860 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68ca9c-9303-4c17-b2dd-123115267c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:40.199741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68ca9c-9303-4c17-b2dd-123115267c60 None admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:40.200021 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68ca9c-9303-4c17-b2dd-123115267c60 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:40.200296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68ca9c-9303-4c17-b2dd-123115267c60 None admin] PATH_INFO: `/v3/system/users/7e33ee3c09ae45e98eda8324e030d4ec/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:40.201923 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68ca9c-9303-4c17-b2dd-123115267c60 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7e33ee3c09ae45e98eda8324e030d4ec, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:40.203339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68ca9c-9303-4c17-b2dd-123115267c60 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:40.204581 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f905e27-1aa8-4cb5-8772-06c9fd5ca69d None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:40.215590 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 151/307] 10.222.0.26 () {58 vars in 1334 bytes} [Mon May 9 09:33:40 2022] PUT /identity/v3/system/users/7e33ee3c09ae45e98eda8324e030d4ec/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:33:40.227317 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d7d3c-2daa-4a96-97cc-f9f345f0ada9 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:40.227681 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d7d3c-2daa-4a96-97cc-f9f345f0ada9 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:40.228016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d7d3c-2daa-4a96-97cc-f9f345f0ada9 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:40.231401 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 152/308] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:33:40.236213 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5c97e-8af9-4dd8-8c93-c44d518bfdf9 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:40.236624 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5c97e-8af9-4dd8-8c93-c44d518bfdf9 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:40.237024 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5c97e-8af9-4dd8-8c93-c44d518bfdf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:40.245712 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91f69b-a4f9-4cd7-b84a-922ccc48566f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:40.246717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91f69b-a4f9-4cd7-b84a-922ccc48566f None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:40.246994 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91f69b-a4f9-4cd7-b84a-922ccc48566f None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:40.247264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91f69b-a4f9-4cd7-b84a-922ccc48566f None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:40.248045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91f69b-a4f9-4cd7-b84a-922ccc48566f None admin] RBAC: Adding query filter params (name=reader) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:40.248349 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91f69b-a4f9-4cd7-b84a-922ccc48566f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:40.250096 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91f69b-a4f9-4cd7-b84a-922ccc48566f None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:40.256573 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-05f5c97e-8af9-4dd8-8c93-c44d518bfdf9 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:40.257842 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-05f5c97e-8af9-4dd8-8c93-c44d518bfdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:40.260200 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 157/309] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:40.265994 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91f69b-a4f9-4cd7-b84a-922ccc48566f None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:40.341167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 153/310] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:33:40.359511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5c97e-8af9-4dd8-8c93-c44d518bfdf9 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:40.374893 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91f69b-a4f9-4cd7-b84a-922ccc48566f None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:40.413632 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f0f5b-d3ae-44cd-80c3-b738508d71ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:40.414160 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f0f5b-d3ae-44cd-80c3-b738508d71ae None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:40.414433 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f0f5b-d3ae-44cd-80c3-b738508d71ae None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:40.414699 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f0f5b-d3ae-44cd-80c3-b738508d71ae None admin] PATH_INFO: `/v3/users/1a063c9951b24110bbcf7004e01376ac` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:40.431189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f0f5b-d3ae-44cd-80c3-b738508d71ae None admin] RBAC: Authorizing `identity:get_user(user_id=1a063c9951b24110bbcf7004e01376ac)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:40.433110 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f0f5b-d3ae-44cd-80c3-b738508d71ae None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:40.435010 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 158/311] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:40 2022] GET /identity/v3/users/1a063c9951b24110bbcf7004e01376ac => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:40.441186 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f0f5b-d3ae-44cd-80c3-b738508d71ae None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:40.473819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5c97e-8af9-4dd8-8c93-c44d518bfdf9 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:40.508680 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e380f39-5bd9-4639-a85b-b5f70c832709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:40.509858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e380f39-5bd9-4639-a85b-b5f70c832709 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:40.510337 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e380f39-5bd9-4639-a85b-b5f70c832709 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:40.510809 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e380f39-5bd9-4639-a85b-b5f70c832709 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:40.521755 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e380f39-5bd9-4639-a85b-b5f70c832709 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:40.529916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-2e380f39-5bd9-4639-a85b-b5f70c832709 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 09:33:40.529916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:40.529916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:40.529916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:40.529916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:40.529916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:40.529916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:40.529916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:40.529916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:40.529916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:40.529916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:40.529916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:40.529916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:40.529916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:40.529916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:40.529916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:40.530799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:40.530799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:40.530799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:40.530799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:40.530799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:40.530799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:40.530799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:40.530799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:40.530799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:40.530799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:40.530799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:40.530799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:33:40.530799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:40.530799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:40.530799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:40.530799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:40.531606 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:40.531606 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:40.531606 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:40.531606 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:40.531606 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:40.531606 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:40.531606 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:40.531606 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:40.531606 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:40.531606 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:40.531606 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 09:33:40.531606 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:33:40.534308 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 154/312] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:40.543289 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e380f39-5bd9-4639-a85b-b5f70c832709 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:40.550770 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f0f5b-d3ae-44cd-80c3-b738508d71ae None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:40.588056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e98641-23de-421a-b891-6370b92c3c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:40.589071 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e98641-23de-421a-b891-6370b92c3c18 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:40.589455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e98641-23de-421a-b891-6370b92c3c18 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:40.589728 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e98641-23de-421a-b891-6370b92c3c18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:40.590843 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e98641-23de-421a-b891-6370b92c3c18 None admin] RBAC: Adding query filter params (role.id=78ec5dc2a67d445389f9a7a4458a12cb, user.id=1a063c9951b24110bbcf7004e01376ac, scope.system=all) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:40.591155 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e98641-23de-421a-b891-6370b92c3c18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:40.592651 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e98641-23de-421a-b891-6370b92c3c18 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:40.608699 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 159/313] 10.222.0.26 () {56 vars in 1308 bytes} [Mon May 9 09:33:40 2022] GET /identity/v3/role_assignments?user.id=1a063c9951b24110bbcf7004e01376ac&scope.system=all&role.id=78ec5dc2a67d445389f9a7a4458a12cb => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:40.645483 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e380f39-5bd9-4639-a85b-b5f70c832709 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:40.682522 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087db18-2949-4634-bb88-444dd5efad93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:40.683467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087db18-2949-4634-bb88-444dd5efad93 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:40.683776 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087db18-2949-4634-bb88-444dd5efad93 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:40.684103 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087db18-2949-4634-bb88-444dd5efad93 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:40.684892 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087db18-2949-4634-bb88-444dd5efad93 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:40.685196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087db18-2949-4634-bb88-444dd5efad93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:40.686610 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087db18-2949-4634-bb88-444dd5efad93 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:40.700702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 155/314] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:40.702154 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e98641-23de-421a-b891-6370b92c3c18 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:40.803702 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e98641-23de-421a-b891-6370b92c3c18 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:40.842043 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0a064-949a-4c92-a7ad-27e6fcac4a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:40.843173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0a064-949a-4c92-a7ad-27e6fcac4a64 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:40.843609 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0a064-949a-4c92-a7ad-27e6fcac4a64 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:40.844099 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0a064-949a-4c92-a7ad-27e6fcac4a64 None admin] PATH_INFO: `/v3/users/2c9c56bff0664840ac81860a642bc08b` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:40.845910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0a064-949a-4c92-a7ad-27e6fcac4a64 None admin] RBAC: Authorizing `identity:get_user(user_id=2c9c56bff0664840ac81860a642bc08b)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:40.847403 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0a064-949a-4c92-a7ad-27e6fcac4a64 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:40.849116 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 160/315] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:40 2022] GET /identity/v3/users/2c9c56bff0664840ac81860a642bc08b => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:40.856241 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087db18-2949-4634-bb88-444dd5efad93 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:40.953520 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087db18-2949-4634-bb88-444dd5efad93 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:40.986267 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f446a4-8757-4aa4-88c9-bbb023b319c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:40.986915 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f446a4-8757-4aa4-88c9-bbb023b319c4 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:40.987193 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f446a4-8757-4aa4-88c9-bbb023b319c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:40.987466 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f446a4-8757-4aa4-88c9-bbb023b319c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:41.054325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f446a4-8757-4aa4-88c9-bbb023b319c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:41.055769 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f446a4-8757-4aa4-88c9-bbb023b319c4 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:41.157398 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 156/316] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:41.165762 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0a064-949a-4c92-a7ad-27e6fcac4a64 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:41.265547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0a064-949a-4c92-a7ad-27e6fcac4a64 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:41.302662 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edc36b-4d58-4a10-ab32-139e93dbc7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:41.303524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edc36b-4d58-4a10-ab32-139e93dbc7bf None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:41.303799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edc36b-4d58-4a10-ab32-139e93dbc7bf None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:41.304068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edc36b-4d58-4a10-ab32-139e93dbc7bf None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:41.305276 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0edc36b-4d58-4a10-ab32-139e93dbc7bf None admin] RBAC: Authorizing `identity:get_project(project_id=9c3d1e7bc10449e2957923e1cbb5e09e)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:41.306734 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0edc36b-4d58-4a10-ab32-139e93dbc7bf None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:41.312686 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 161/317] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:41 2022] GET /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:41.314663 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f446a4-8757-4aa4-88c9-bbb023b319c4 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:41.419243 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f446a4-8757-4aa4-88c9-bbb023b319c4 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:41.452937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c97696-11ad-4aab-b9e1-7fd2a24840bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:41.453592 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c97696-11ad-4aab-b9e1-7fd2a24840bc None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:41.453865 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c97696-11ad-4aab-b9e1-7fd2a24840bc None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:41.454133 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c97696-11ad-4aab-b9e1-7fd2a24840bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:41.454962 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c97696-11ad-4aab-b9e1-7fd2a24840bc None admin] RBAC: Adding query filter params (role.id=78ec5dc2a67d445389f9a7a4458a12cb, scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e, user.id=2c9c56bff0664840ac81860a642bc08b) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:41.455266 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c97696-11ad-4aab-b9e1-7fd2a24840bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:41.456728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c97696-11ad-4aab-b9e1-7fd2a24840bc None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:41.462795 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 157/318] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:33:41 2022] GET /identity/v3/role_assignments?user.id=2c9c56bff0664840ac81860a642bc08b&scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e&role.id=78ec5dc2a67d445389f9a7a4458a12cb => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:42.467990 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7faeb3-dc80-40d0-b33a-3b362582d50b None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:42.468543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7faeb3-dc80-40d0-b33a-3b362582d50b None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:42.468543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7faeb3-dc80-40d0-b33a-3b362582d50b None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:42.469905 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 162/319] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:42.475511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba400720-b79f-46f5-82a0-0e05234ef5d8 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:42.475943 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba400720-b79f-46f5-82a0-0e05234ef5d8 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:42.476016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba400720-b79f-46f5-82a0-0e05234ef5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:42.493173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-ba400720-b79f-46f5-82a0-0e05234ef5d8 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:42.494359 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-ba400720-b79f-46f5-82a0-0e05234ef5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:42.507506 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb023aba-d813-49ea-8a1c-30cb6e08c9b1 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:42.507786 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb023aba-d813-49ea-8a1c-30cb6e08c9b1 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:42.508052 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb023aba-d813-49ea-8a1c-30cb6e08c9b1 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:42.509480 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 163/320] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:42.514506 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e347554-725a-4af6-b4a0-246d101d26ff None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:42.514761 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e347554-725a-4af6-b4a0-246d101d26ff None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:42.515025 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e347554-725a-4af6-b4a0-246d101d26ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:42.531608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-4e347554-725a-4af6-b4a0-246d101d26ff None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:42.533108 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-4e347554-725a-4af6-b4a0-246d101d26ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:42.568722 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 158/321] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:42.590788 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba400720-b79f-46f5-82a0-0e05234ef5d8 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:42.598780 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 164/322] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:33:42.621596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e347554-725a-4af6-b4a0-246d101d26ff None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:42.686847 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba400720-b79f-46f5-82a0-0e05234ef5d8 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:42.718925 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e347554-725a-4af6-b4a0-246d101d26ff None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:42.726955 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17c55b-79b9-4190-bb21-e5b932350fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:42.727920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17c55b-79b9-4190-bb21-e5b932350fea None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:42.728334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17c55b-79b9-4190-bb21-e5b932350fea None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:42.728765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17c55b-79b9-4190-bb21-e5b932350fea None admin] PATH_INFO: `/v3/domains/default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:42.730109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa17c55b-79b9-4190-bb21-e5b932350fea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:42.731694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa17c55b-79b9-4190-bb21-e5b932350fea None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:42.733804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 159/323] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:33:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:42.738993 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17c55b-79b9-4190-bb21-e5b932350fea None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:42.750540 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efdefc-8fb1-4dbf-8823-f4883b9b4017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:42.751326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efdefc-8fb1-4dbf-8823-f4883b9b4017 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:42.751602 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efdefc-8fb1-4dbf-8823-f4883b9b4017 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:42.751870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efdefc-8fb1-4dbf-8823-f4883b9b4017 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:42.761734 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1efdefc-8fb1-4dbf-8823-f4883b9b4017 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:42.769338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-a1efdefc-8fb1-4dbf-8823-f4883b9b4017 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:42.769338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:42.769338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:42.769338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:42.769338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:42.769338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:42.769338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:42.769338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:42.769338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:42.769338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:42.769338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:42.769338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:42.769338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:42.769338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:42.769338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:42.769338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:42.770215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:42.770215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:42.770215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:42.770215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:42.770215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:42.770215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:42.770215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:42.770215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:42.770215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:42.770215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:42.770215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:42.770215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:42.770215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:42.770215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:42.770215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:42.770215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:42.776045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:42.776045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:42.776045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:42.776045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:42.776045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:42.776045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:42.776045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:42.776045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:42.776045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:42.776045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:42.776045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:42.776045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:42.776045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 165/324] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:33:42.777238 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efdefc-8fb1-4dbf-8823-f4883b9b4017 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:42.855255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17c55b-79b9-4190-bb21-e5b932350fea None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:42.873546 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efdefc-8fb1-4dbf-8823-f4883b9b4017 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:42.888492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98189b5f-b424-4325-a713-d9172e9493cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:42.889197 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98189b5f-b424-4325-a713-d9172e9493cf None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:42.889449 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98189b5f-b424-4325-a713-d9172e9493cf None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:42.889715 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98189b5f-b424-4325-a713-d9172e9493cf None admin] PATH_INFO: `/v3/groups` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:42.890410 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98189b5f-b424-4325-a713-d9172e9493cf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:42.891876 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98189b5f-b424-4325-a713-d9172e9493cf None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:42.906453 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba1fdd-349e-486c-b1f9-19915137f46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:42.907099 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba1fdd-349e-486c-b1f9-19915137f46c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:42.907374 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba1fdd-349e-486c-b1f9-19915137f46c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:42.907846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba1fdd-349e-486c-b1f9-19915137f46c None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:42.907974 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 160/325] 10.222.0.26 () {60 vars in 1134 bytes} [Mon May 9 09:33:42 2022] POST /identity/v3/groups => generated 226 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:33:42.908411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba1fdd-349e-486c-b1f9-19915137f46c None admin] RBAC: Adding query filter params (name=member) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:42.908705 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba1fdd-349e-486c-b1f9-19915137f46c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:42.909000 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c157bc-2819-464a-8280-ec8cb4f5c4f0 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:42.909277 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c157bc-2819-464a-8280-ec8cb4f5c4f0 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:42.909549 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c157bc-2819-464a-8280-ec8cb4f5c4f0 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:42.910107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba1fdd-349e-486c-b1f9-19915137f46c None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:42.912718 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 161/326] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:42.916949 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e1ffb-f5bf-43f3-bdb8-29b289e0491b None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:42.918112 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e1ffb-f5bf-43f3-bdb8-29b289e0491b None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:42.918369 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e1ffb-f5bf-43f3-bdb8-29b289e0491b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:42.925854 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 166/327] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:42.927048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba1fdd-349e-486c-b1f9-19915137f46c None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:42.936607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-b77e1ffb-f5bf-43f3-bdb8-29b289e0491b None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:42.937879 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-b77e1ffb-f5bf-43f3-bdb8-29b289e0491b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:43.010696 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 162/328] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:43.031831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba1fdd-349e-486c-b1f9-19915137f46c None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:43.032417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e1ffb-f5bf-43f3-bdb8-29b289e0491b None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:43.065956 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264a83b-3156-4e30-82ce-81d0c53ac46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:43.066829 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264a83b-3156-4e30-82ce-81d0c53ac46e None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:43.067103 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264a83b-3156-4e30-82ce-81d0c53ac46e None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:43.067388 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264a83b-3156-4e30-82ce-81d0c53ac46e None admin] PATH_INFO: `/v3/users/7e33ee3c09ae45e98eda8324e030d4ec` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:43.068702 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264a83b-3156-4e30-82ce-81d0c53ac46e None admin] RBAC: Authorizing `identity:get_user(user_id=7e33ee3c09ae45e98eda8324e030d4ec)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:43.070167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264a83b-3156-4e30-82ce-81d0c53ac46e None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:43.071847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 167/329] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:42 2022] GET /identity/v3/users/7e33ee3c09ae45e98eda8324e030d4ec => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:43.077886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264a83b-3156-4e30-82ce-81d0c53ac46e None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:43.134584 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e1ffb-f5bf-43f3-bdb8-29b289e0491b None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:43.170861 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4b6d3-c315-4da9-9cf1-2955f32754a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:43.171664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4b6d3-c315-4da9-9cf1-2955f32754a4 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:43.172993 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4b6d3-c315-4da9-9cf1-2955f32754a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:43.172993 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4b6d3-c315-4da9-9cf1-2955f32754a4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:43.176468 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264a83b-3156-4e30-82ce-81d0c53ac46e None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:43.189021 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c4b6d3-c315-4da9-9cf1-2955f32754a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:43.196182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-24c4b6d3-c315-4da9-9cf1-2955f32754a4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 09:33:43.196182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:43.196182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:43.196182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:43.196182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:43.196182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:43.196182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:43.196182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:43.196182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:43.196182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:43.196182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:43.196182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:43.196182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:43.196182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:43.196182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:43.196182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:43.197323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:43.197323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:43.197323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:43.197323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:43.197323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:43.197323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:43.197323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:43.197323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:43.197323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:43.197323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:43.197323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:43.197323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:33:43.197323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:43.197323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:43.197323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:43.197323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:43.198413 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:43.198413 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:43.198413 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:43.198413 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:43.198413 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:43.198413 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:43.198413 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:43.198413 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:43.198413 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:43.198413 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:43.198413 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 09:33:43.198413 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:33:43.198413 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 163/330] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:43.204302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4b6d3-c315-4da9-9cf1-2955f32754a4 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:43.210372 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7103f3-c2b9-43c5-9b7f-ae06d32e43ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:43.211025 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7103f3-c2b9-43c5-9b7f-ae06d32e43ae None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:43.211298 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7103f3-c2b9-43c5-9b7f-ae06d32e43ae None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:43.211562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7103f3-c2b9-43c5-9b7f-ae06d32e43ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:43.212395 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7103f3-c2b9-43c5-9b7f-ae06d32e43ae None admin] RBAC: Adding query filter params (role.id=fec647282f5a42ab8490975708efad14, user.id=7e33ee3c09ae45e98eda8324e030d4ec, scope.system=all) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:43.212726 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7103f3-c2b9-43c5-9b7f-ae06d32e43ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:43.214282 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7103f3-c2b9-43c5-9b7f-ae06d32e43ae None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:43.226586 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 168/331] 10.222.0.26 () {56 vars in 1308 bytes} [Mon May 9 09:33:43 2022] GET /identity/v3/role_assignments?user.id=7e33ee3c09ae45e98eda8324e030d4ec&scope.system=all&role.id=fec647282f5a42ab8490975708efad14 => generated 510 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:43.327399 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4b6d3-c315-4da9-9cf1-2955f32754a4 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:43.365918 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a8f24-b386-478c-a58d-64b17bce4de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:43.366784 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a8f24-b386-478c-a58d-64b17bce4de1 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:43.367242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a8f24-b386-478c-a58d-64b17bce4de1 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:43.367623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a8f24-b386-478c-a58d-64b17bce4de1 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:43.368480 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a8f24-b386-478c-a58d-64b17bce4de1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:43.368876 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a8f24-b386-478c-a58d-64b17bce4de1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:43.370444 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a8f24-b386-478c-a58d-64b17bce4de1 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:43.380790 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 164/332] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:43.387198 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7103f3-c2b9-43c5-9b7f-ae06d32e43ae None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:43.496847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7103f3-c2b9-43c5-9b7f-ae06d32e43ae None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:43.538347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4bbb8-a3a1-4911-a4b5-7fefee85ba3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:43.539240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4bbb8-a3a1-4911-a4b5-7fefee85ba3c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:43.539512 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4bbb8-a3a1-4911-a4b5-7fefee85ba3c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:43.539778 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4bbb8-a3a1-4911-a4b5-7fefee85ba3c None admin] PATH_INFO: `/v3/users/1a063c9951b24110bbcf7004e01376ac` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:43.549089 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb4bbb8-a3a1-4911-a4b5-7fefee85ba3c None admin] RBAC: Authorizing `identity:get_user(user_id=1a063c9951b24110bbcf7004e01376ac)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:43.550559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb4bbb8-a3a1-4911-a4b5-7fefee85ba3c None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:43.552562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 169/333] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:43 2022] GET /identity/v3/users/1a063c9951b24110bbcf7004e01376ac => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:43.561535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a8f24-b386-478c-a58d-64b17bce4de1 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:43.669251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a8f24-b386-478c-a58d-64b17bce4de1 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:43.705306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd87b6c-3357-4cc8-9157-48d02eae066b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:43.706427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd87b6c-3357-4cc8-9157-48d02eae066b None admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:43.706903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd87b6c-3357-4cc8-9157-48d02eae066b None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:43.707342 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd87b6c-3357-4cc8-9157-48d02eae066b None admin] PATH_INFO: `/v3/system/users/1a063c9951b24110bbcf7004e01376ac/roles/78ec5dc2a67d445389f9a7a4458a12cb` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:43.709076 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd87b6c-3357-4cc8-9157-48d02eae066b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1a063c9951b24110bbcf7004e01376ac, role_id=78ec5dc2a67d445389f9a7a4458a12cb)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:43.710640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd87b6c-3357-4cc8-9157-48d02eae066b None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:43.727458 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 165/334] 10.222.0.26 () {58 vars in 1334 bytes} [Mon May 9 09:33:43 2022] PUT /identity/v3/system/users/1a063c9951b24110bbcf7004e01376ac/roles/78ec5dc2a67d445389f9a7a4458a12cb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:33:45.446430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c93e82-a8af-4dcf-b7fe-7a8944776ad4 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:45.446959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c93e82-a8af-4dcf-b7fe-7a8944776ad4 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:45.446959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c93e82-a8af-4dcf-b7fe-7a8944776ad4 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:45.448374 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 170/335] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:45.454320 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf1a70-8cbf-486f-a52d-1abc085828fa None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:45.454773 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf1a70-8cbf-486f-a52d-1abc085828fa None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:45.454850 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf1a70-8cbf-486f-a52d-1abc085828fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:45.472093 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-10bf1a70-8cbf-486f-a52d-1abc085828fa None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:45.473323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-10bf1a70-8cbf-486f-a52d-1abc085828fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:45.544696 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 166/336] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:33:45.566674 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c93e82-a8af-4dcf-b7fe-7a8944776ad4 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:45.686366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c93e82-a8af-4dcf-b7fe-7a8944776ad4 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:45.723897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42acc57-75e2-4aff-ac26-26ec58664c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:45.724618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42acc57-75e2-4aff-ac26-26ec58664c70 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:45.724922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42acc57-75e2-4aff-ac26-26ec58664c70 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:45.725191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42acc57-75e2-4aff-ac26-26ec58664c70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:45.735267 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42acc57-75e2-4aff-ac26-26ec58664c70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:45.743179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-f42acc57-75e2-4aff-ac26-26ec58664c70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:45.743179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:45.743179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:45.743179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:45.743179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:45.743179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:45.743179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:45.743179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:45.743179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:45.743179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:45.743179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:45.743179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:45.743179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:45.743179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:45.743179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:45.743179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:45.744164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:45.744164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:45.744164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:45.744164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:45.744164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:45.744164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:45.744164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:45.744164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:45.744164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:45.744164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:45.744164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:45.744164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:45.744164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:45.744164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:45.744164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:45.744164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:45.745148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:45.745148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:45.745148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:45.745148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:45.745148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:45.745148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:45.745148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:45.745148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:45.745148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:45.745148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:45.745148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:45.745148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:45.745148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 171/337] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:45.750944 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf1a70-8cbf-486f-a52d-1abc085828fa None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:45.851095 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf1a70-8cbf-486f-a52d-1abc085828fa None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:45.888201 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c3c8a-86dd-4710-8cfb-419ba039764f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:45.888923 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c3c8a-86dd-4710-8cfb-419ba039764f None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:45.889197 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c3c8a-86dd-4710-8cfb-419ba039764f None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:45.889462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c3c8a-86dd-4710-8cfb-419ba039764f None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:45.890510 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c3c8a-86dd-4710-8cfb-419ba039764f None admin] RBAC: Adding query filter params (name=member) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:45.890815 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c3c8a-86dd-4710-8cfb-419ba039764f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:45.892233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c3c8a-86dd-4710-8cfb-419ba039764f None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:45.902566 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 167/338] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:45.909156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42acc57-75e2-4aff-ac26-26ec58664c70 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:46.011222 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42acc57-75e2-4aff-ac26-26ec58664c70 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:46.044480 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbdc06-1b91-4a85-8b86-12f0d2dd4c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:46.045210 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbdc06-1b91-4a85-8b86-12f0d2dd4c8a None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:46.045486 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbdc06-1b91-4a85-8b86-12f0d2dd4c8a None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:46.045758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbdc06-1b91-4a85-8b86-12f0d2dd4c8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:46.112834 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbdc06-1b91-4a85-8b86-12f0d2dd4c8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:46.114414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbdc06-1b91-4a85-8b86-12f0d2dd4c8a None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:46.167985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed14a5-9abf-4e6b-b36b-4d1a72a27ba6 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:46.168340 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed14a5-9abf-4e6b-b36b-4d1a72a27ba6 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:46.168511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed14a5-9abf-4e6b-b36b-4d1a72a27ba6 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:46.170090 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 168/339] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:46.175577 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5234a-6d89-4060-97be-3e4fb339ee56 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:46.175858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5234a-6d89-4060-97be-3e4fb339ee56 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:46.176132 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5234a-6d89-4060-97be-3e4fb339ee56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:46.203859 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-d5c5234a-6d89-4060-97be-3e4fb339ee56 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:46.206771 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-d5c5234a-6d89-4060-97be-3e4fb339ee56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:46.259809 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 172/340] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:46.265513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbdc06-1b91-4a85-8b86-12f0d2dd4c8a None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:46.383157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbdc06-1b91-4a85-8b86-12f0d2dd4c8a None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:46.402173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 169/341] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:46.417748 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18750d-9ac1-46d4-bc8b-b3f2bb3ed5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:46.418668 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18750d-9ac1-46d4-bc8b-b3f2bb3ed5a7 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:46.418944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18750d-9ac1-46d4-bc8b-b3f2bb3ed5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:46.419218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18750d-9ac1-46d4-bc8b-b3f2bb3ed5a7 None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:46.420436 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18750d-9ac1-46d4-bc8b-b3f2bb3ed5a7 None admin] RBAC: Authorizing `identity:get_project(project_id=9ec214e8e5724ce6b35fe31337cb0380)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:46.421938 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18750d-9ac1-46d4-bc8b-b3f2bb3ed5a7 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:46.424051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 173/342] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:46 2022] GET /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:46.429531 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5234a-6d89-4060-97be-3e4fb339ee56 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:46.451361 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18750d-9ac1-46d4-bc8b-b3f2bb3ed5a7 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:46.566021 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5234a-6d89-4060-97be-3e4fb339ee56 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:46.594517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18750d-9ac1-46d4-bc8b-b3f2bb3ed5a7 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:46.603470 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891c06a-7e6d-43b9-80fe-c11a9705a40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:46.604655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891c06a-7e6d-43b9-80fe-c11a9705a40a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:46.605235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891c06a-7e6d-43b9-80fe-c11a9705a40a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:46.605685 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891c06a-7e6d-43b9-80fe-c11a9705a40a None admin] PATH_INFO: `/v3/groups/1ff36c94c8f6421da190b330cdbd9185` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:46.613816 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c891c06a-7e6d-43b9-80fe-c11a9705a40a None admin] RBAC: Authorizing `identity:get_group(group_id=1ff36c94c8f6421da190b330cdbd9185)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:46.615800 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c891c06a-7e6d-43b9-80fe-c11a9705a40a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:46.618378 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 170/343] 10.222.0.26 () {56 vars in 1180 bytes} [Mon May 9 09:33:46 2022] GET /identity/v3/groups/1ff36c94c8f6421da190b330cdbd9185 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:46.620117 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2480cd7-3189-45dd-9fc2-a35b21be8af5 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:46.620586 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2480cd7-3189-45dd-9fc2-a35b21be8af5 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:46.621373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2480cd7-3189-45dd-9fc2-a35b21be8af5 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:46.627570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 171/344] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:46.629398 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008d637-fa34-4185-b26e-df0204fd634a None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:46.629883 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008d637-fa34-4185-b26e-df0204fd634a None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:46.630327 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008d637-fa34-4185-b26e-df0204fd634a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:46.649241 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d68d3f-4860-4632-ab91-5aec4f697dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:46.649716 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-9008d637-fa34-4185-b26e-df0204fd634a None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:46.650330 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d68d3f-4860-4632-ab91-5aec4f697dbc None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:46.650678 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d68d3f-4860-4632-ab91-5aec4f697dbc None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:46.651016 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d68d3f-4860-4632-ab91-5aec4f697dbc None admin] PATH_INFO: `/v3/roles/member` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:46.651719 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-9008d637-fa34-4185-b26e-df0204fd634a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:46.666364 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d68d3f-4860-4632-ab91-5aec4f697dbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:46.674430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-89d68d3f-4860-4632-ab91-5aec4f697dbc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:46.674430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:46.674430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:46.674430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:46.674430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:46.674430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:46.674430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:46.674430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:46.674430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:46.674430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:46.674430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:46.674430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:46.674430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:46.674430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:46.674430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:46.674430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:46.675338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:46.675338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:46.675338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:46.675338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:46.675338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:46.675338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:46.675338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:46.675338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:46.675338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:46.675338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:46.675338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:46.675338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:46.675338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:46.675338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:46.675338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:46.675338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:46.676216 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:46.676216 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:46.676216 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:46.676216 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:46.676216 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:46.676216 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:46.676216 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:46.676216 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:46.676216 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:46.676216 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:46.676216 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:46.676216 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:46.676216 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 174/345] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:33:46.681039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d68d3f-4860-4632-ab91-5aec4f697dbc None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:46.763130 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 172/346] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:46.764271 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9008d637-fa34-4185-b26e-df0204fd634a None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:46.780766 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d68d3f-4860-4632-ab91-5aec4f697dbc None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:46.819319 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ba799-8cb8-4928-a299-07db0c333c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:46.819965 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ba799-8cb8-4928-a299-07db0c333c85 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:46.820338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ba799-8cb8-4928-a299-07db0c333c85 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:46.820836 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ba799-8cb8-4928-a299-07db0c333c85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:46.821655 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ba799-8cb8-4928-a299-07db0c333c85 None admin] RBAC: Adding query filter params (group.id=1ff36c94c8f6421da190b330cdbd9185, role.id=fec647282f5a42ab8490975708efad14, scope.project.id=9ec214e8e5724ce6b35fe31337cb0380) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:46.822000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ba799-8cb8-4928-a299-07db0c333c85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:46.823605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ba799-8cb8-4928-a299-07db0c333c85 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:46.829583 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 175/347] 10.222.0.26 () {56 vars in 1376 bytes} [Mon May 9 09:33:46 2022] GET /identity/v3/role_assignments?group.id=1ff36c94c8f6421da190b330cdbd9185&scope.project.id=9ec214e8e5724ce6b35fe31337cb0380&role.id=fec647282f5a42ab8490975708efad14 => generated 260 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:46.830468 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ba799-8cb8-4928-a299-07db0c333c85 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:46.890530 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9008d637-fa34-4185-b26e-df0204fd634a None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:46.935806 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b0a50-8de0-490e-8912-c367a2e2637f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:46.936928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b0a50-8de0-490e-8912-c367a2e2637f None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:46.937406 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b0a50-8de0-490e-8912-c367a2e2637f None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:46.939265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b0a50-8de0-490e-8912-c367a2e2637f None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:46.940276 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b0a50-8de0-490e-8912-c367a2e2637f None admin] RBAC: Adding query filter params (name=member) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:46.940807 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b0a50-8de0-490e-8912-c367a2e2637f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:46.942934 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b0a50-8de0-490e-8912-c367a2e2637f None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:46.961611 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 173/348] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:46.965585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2874c-2f17-4cc8-876b-17e31018807a None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:46.966834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2874c-2f17-4cc8-876b-17e31018807a None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:46.967493 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2874c-2f17-4cc8-876b-17e31018807a None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:46.971193 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 174/349] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:46.972944 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2874c-2f17-4cc8-876b-17e31018807a None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:46.982766 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ba799-8cb8-4928-a299-07db0c333c85 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:47.065894 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94acff-ce28-4b19-98aa-f63087a0f757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:47.066728 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94acff-ce28-4b19-98aa-f63087a0f757 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:47.067006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94acff-ce28-4b19-98aa-f63087a0f757 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:47.067278 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94acff-ce28-4b19-98aa-f63087a0f757 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:47.094791 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b94acff-ce28-4b19-98aa-f63087a0f757 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:47.098408 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2874c-2f17-4cc8-876b-17e31018807a None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:47.103214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-1b94acff-ce28-4b19-98aa-f63087a0f757 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:47.103214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:47.103214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:47.103214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:47.103214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:47.103214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:47.103214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:47.103214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:47.103214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:47.103214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:47.103214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:47.103214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:47.103214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:47.103214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:47.103214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:47.103214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:47.104215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:47.104215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:47.104215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:47.104215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:47.104215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:47.104215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:47.104215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:47.104215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:47.104215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:47.104215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:47.104215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:47.104215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:47.104215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:47.104215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:47.104215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:47.104215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:47.105561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:47.105561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:47.105561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:47.105561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:47.105561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:47.105561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:47.105561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:47.105561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:47.105561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:47.105561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:47.105561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:47.105561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:47.105561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 176/350] 10.222.0.26 () {56 vars in 1114 bytes} [Mon May 9 09:33:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:47.106308 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67b0cc-4a29-4909-9bac-cb051c2c3660 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:47.106308 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67b0cc-4a29-4909-9bac-cb051c2c3660 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:47.106740 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67b0cc-4a29-4909-9bac-cb051c2c3660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:47.130776 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-ef67b0cc-4a29-4909-9bac-cb051c2c3660 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:47.131973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-ef67b0cc-4a29-4909-9bac-cb051c2c3660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:47.142076 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091ae63-b73a-4ce3-beff-25b61c69840b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:47.143587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091ae63-b73a-4ce3-beff-25b61c69840b None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:47.144076 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091ae63-b73a-4ce3-beff-25b61c69840b None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:47.144553 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091ae63-b73a-4ce3-beff-25b61c69840b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:47.207573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 177/351] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:47.208440 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67b0cc-4a29-4909-9bac-cb051c2c3660 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:47.231239 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0091ae63-b73a-4ce3-beff-25b61c69840b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:47.233407 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0091ae63-b73a-4ce3-beff-25b61c69840b None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:47.334521 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67b0cc-4a29-4909-9bac-cb051c2c3660 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:47.362674 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 175/352] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:47.364376 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091ae63-b73a-4ce3-beff-25b61c69840b None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:47.368451 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6271db-110b-47d3-b6b1-5803dcd0b881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:47.368917 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6271db-110b-47d3-b6b1-5803dcd0b881 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:47.369291 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6271db-110b-47d3-b6b1-5803dcd0b881 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:47.369660 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6271db-110b-47d3-b6b1-5803dcd0b881 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:47.370562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6271db-110b-47d3-b6b1-5803dcd0b881 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:47.371279 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6271db-110b-47d3-b6b1-5803dcd0b881 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:47.372797 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6271db-110b-47d3-b6b1-5803dcd0b881 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:47.381379 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 178/353] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:33:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:47.382132 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6271db-110b-47d3-b6b1-5803dcd0b881 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:47.497501 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091ae63-b73a-4ce3-beff-25b61c69840b None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:47.521363 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6271db-110b-47d3-b6b1-5803dcd0b881 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:47.542929 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b0a1c-2b07-4ef6-8b87-3d397dd3078c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:47.544006 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b0a1c-2b07-4ef6-8b87-3d397dd3078c None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:47.544434 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b0a1c-2b07-4ef6-8b87-3d397dd3078c None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:47.544892 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b0a1c-2b07-4ef6-8b87-3d397dd3078c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:47.562461 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b0a1c-2b07-4ef6-8b87-3d397dd3078c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:47.567661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19da03c-1bb6-4ddc-9498-1dc7ff824ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:47.568832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19da03c-1bb6-4ddc-9498-1dc7ff824ff1 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:47.569324 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19da03c-1bb6-4ddc-9498-1dc7ff824ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:47.569803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19da03c-1bb6-4ddc-9498-1dc7ff824ff1 None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:47.570325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-5f4b0a1c-2b07-4ef6-8b87-3d397dd3078c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 09:33:47.570325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:47.570325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:47.570325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:47.570325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:47.570325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:47.570325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:47.570325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:47.570325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:47.570325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:47.570325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:47.570325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:47.570325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:47.570325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:47.570325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:47.570325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:47.571471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:47.571471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:47.571471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:47.571471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:47.571471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:47.571471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:47.571471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:47.571471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:47.571471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:47.571471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:47.571471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:47.571471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:33:47.571471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:47.571471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:47.571471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:47.571471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:47.572263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:47.572263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:47.572263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:47.572263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:47.572263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:47.572263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:47.572263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:47.572263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:47.572263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:47.572263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:47.572263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 09:33:47.572263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:33:47.572263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 176/354] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:47.573109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b0a1c-2b07-4ef6-8b87-3d397dd3078c None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:47.578770 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19da03c-1bb6-4ddc-9498-1dc7ff824ff1 None admin] RBAC: Authorizing `identity:get_project(project_id=9c3d1e7bc10449e2957923e1cbb5e09e)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:47.580575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19da03c-1bb6-4ddc-9498-1dc7ff824ff1 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:47.582910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 179/355] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:47 2022] GET /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:47.584402 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b684b-72a4-47a6-b519-48debbe9e271 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:47.584929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b684b-72a4-47a6-b519-48debbe9e271 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:47.585414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b684b-72a4-47a6-b519-48debbe9e271 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:47.587063 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 180/356] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:47.588005 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b684b-72a4-47a6-b519-48debbe9e271 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:47.679614 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b0a1c-2b07-4ef6-8b87-3d397dd3078c None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:47.690959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b684b-72a4-47a6-b519-48debbe9e271 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:47.713409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6884e6-a978-492d-95db-06f4f44e5b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:47.714375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6884e6-a978-492d-95db-06f4f44e5b1a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:47.714513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6884e6-a978-492d-95db-06f4f44e5b1a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:47.714691 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6884e6-a978-492d-95db-06f4f44e5b1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:47.728460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1f5ba-4fb6-4563-ab7c-beecc20ca018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:47.729559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1f5ba-4fb6-4563-ab7c-beecc20ca018 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:47.730086 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1f5ba-4fb6-4563-ab7c-beecc20ca018 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:47.730566 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1f5ba-4fb6-4563-ab7c-beecc20ca018 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:47.731603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf1f5ba-4fb6-4563-ab7c-beecc20ca018 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:47.732102 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf1f5ba-4fb6-4563-ab7c-beecc20ca018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:47.733876 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf1f5ba-4fb6-4563-ab7c-beecc20ca018 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:47.742721 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 181/357] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:47.743830 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1f5ba-4fb6-4563-ab7c-beecc20ca018 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:47.790320 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6884e6-a978-492d-95db-06f4f44e5b1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:47.791844 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6884e6-a978-492d-95db-06f4f44e5b1a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:47.842726 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1f5ba-4fb6-4563-ab7c-beecc20ca018 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:47.876391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b02d0-221f-410e-952e-856830e90020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:47.877682 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b02d0-221f-410e-952e-856830e90020 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:47.878155 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b02d0-221f-410e-952e-856830e90020 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:47.878647 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b02d0-221f-410e-952e-856830e90020 None admin] PATH_INFO: `/v3/groups/1ff36c94c8f6421da190b330cdbd9185` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:47.880061 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b02d0-221f-410e-952e-856830e90020 None admin] RBAC: Authorizing `identity:get_group(group_id=1ff36c94c8f6421da190b330cdbd9185)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:47.881770 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b02d0-221f-410e-952e-856830e90020 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:47.883752 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 182/358] 10.222.0.26 () {56 vars in 1180 bytes} [Mon May 9 09:33:47 2022] GET /identity/v3/groups/1ff36c94c8f6421da190b330cdbd9185 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:47.885120 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2611e-ff52-4c4d-a249-e9e4da46900e None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:47.885742 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2611e-ff52-4c4d-a249-e9e4da46900e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:47.886484 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2611e-ff52-4c4d-a249-e9e4da46900e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:47.891603 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 177/359] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:47.892466 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6884e6-a978-492d-95db-06f4f44e5b1a None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:47.907992 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-baf2611e-ff52-4c4d-a249-e9e4da46900e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:47.909635 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-baf2611e-ff52-4c4d-a249-e9e4da46900e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:47.978341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 183/360] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:47.979586 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2611e-ff52-4c4d-a249-e9e4da46900e None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:47.997334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6884e6-a978-492d-95db-06f4f44e5b1a None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:48.032456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d417106a-4f9c-4b3d-bd78-495fdf863187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:48.033364 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417106a-4f9c-4b3d-bd78-495fdf863187 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:48.033646 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417106a-4f9c-4b3d-bd78-495fdf863187 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:48.033918 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417106a-4f9c-4b3d-bd78-495fdf863187 None admin] PATH_INFO: `/v3/users/1a063c9951b24110bbcf7004e01376ac` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:48.035999 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d417106a-4f9c-4b3d-bd78-495fdf863187 None admin] RBAC: Authorizing `identity:get_user(user_id=1a063c9951b24110bbcf7004e01376ac)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:48.037504 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d417106a-4f9c-4b3d-bd78-495fdf863187 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:48.039166 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 178/361] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:33:47 2022] GET /identity/v3/users/1a063c9951b24110bbcf7004e01376ac => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:48.039917 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d417106a-4f9c-4b3d-bd78-495fdf863187 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:48.082234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2611e-ff52-4c4d-a249-e9e4da46900e None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:48.119575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ecacf-e624-4db2-b446-9d702a48df2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:48.120648 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ecacf-e624-4db2-b446-9d702a48df2b None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:48.121460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ecacf-e624-4db2-b446-9d702a48df2b None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:48.121920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ecacf-e624-4db2-b446-9d702a48df2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:48.123034 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ecacf-e624-4db2-b446-9d702a48df2b None admin] RBAC: Adding query filter params (group.id=1ff36c94c8f6421da190b330cdbd9185, role.id=fec647282f5a42ab8490975708efad14, scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:48.123552 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ecacf-e624-4db2-b446-9d702a48df2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:48.125343 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ecacf-e624-4db2-b446-9d702a48df2b None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:48.131622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 184/362] 10.222.0.26 () {56 vars in 1376 bytes} [Mon May 9 09:33:47 2022] GET /identity/v3/role_assignments?group.id=1ff36c94c8f6421da190b330cdbd9185&scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e&role.id=fec647282f5a42ab8490975708efad14 => generated 260 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:48.132656 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ecacf-e624-4db2-b446-9d702a48df2b None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:48.151258 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d417106a-4f9c-4b3d-bd78-495fdf863187 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:48.188358 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349a31b-26f7-431d-a34a-c5ae35412a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:48.189285 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349a31b-26f7-431d-a34a-c5ae35412a51 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:48.189564 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349a31b-26f7-431d-a34a-c5ae35412a51 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:48.189837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349a31b-26f7-431d-a34a-c5ae35412a51 None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:48.191056 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349a31b-26f7-431d-a34a-c5ae35412a51 None admin] RBAC: Authorizing `identity:get_project(project_id=9ec214e8e5724ce6b35fe31337cb0380)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:48.192535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349a31b-26f7-431d-a34a-c5ae35412a51 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:48.194430 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 179/363] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:48 2022] GET /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:48.195044 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349a31b-26f7-431d-a34a-c5ae35412a51 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:48.244558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ecacf-e624-4db2-b446-9d702a48df2b None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:48.282470 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c508a8-5781-4717-91ed-a06b97c496ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:48.283750 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c508a8-5781-4717-91ed-a06b97c496ac None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:48.284244 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c508a8-5781-4717-91ed-a06b97c496ac None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:48.284782 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c508a8-5781-4717-91ed-a06b97c496ac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:48.296143 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c508a8-5781-4717-91ed-a06b97c496ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:48.304883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-44c508a8-5781-4717-91ed-a06b97c496ac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:48.304883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:48.304883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:48.304883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:48.304883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:48.304883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:48.304883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:48.304883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:48.304883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:48.304883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:48.304883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:48.304883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:48.304883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:48.304883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:48.304883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:48.304883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:48.305860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:48.305860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:48.305860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:48.305860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:48.305860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:48.305860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:48.305860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:48.305860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:48.305860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:48.305860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:48.305860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:48.305860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:48.305860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:48.305860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:48.305860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:48.305860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:48.306662 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:48.306662 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:48.306662 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:48.306662 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:48.306662 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:48.306662 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:48.306662 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:48.306662 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:48.306662 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:48.306662 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:48.306662 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:48.306662 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:48.309540 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 185/364] 10.222.0.26 () {56 vars in 1114 bytes} [Mon May 9 09:33:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:48.310609 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c508a8-5781-4717-91ed-a06b97c496ac None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:48.333429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349a31b-26f7-431d-a34a-c5ae35412a51 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:48.369908 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cca290-9333-4f6a-9ba3-ec8621bbebe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:48.370831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cca290-9333-4f6a-9ba3-ec8621bbebe2 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:48.370831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cca290-9333-4f6a-9ba3-ec8621bbebe2 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:48.371471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cca290-9333-4f6a-9ba3-ec8621bbebe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:48.372264 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cca290-9333-4f6a-9ba3-ec8621bbebe2 None admin] RBAC: Adding query filter params (role.id=78ec5dc2a67d445389f9a7a4458a12cb, user.id=1a063c9951b24110bbcf7004e01376ac, scope.system=all) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:48.372575 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cca290-9333-4f6a-9ba3-ec8621bbebe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:48.374091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cca290-9333-4f6a-9ba3-ec8621bbebe2 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:48.388321 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 180/365] 10.222.0.26 () {56 vars in 1308 bytes} [Mon May 9 09:33:48 2022] GET /identity/v3/role_assignments?user.id=1a063c9951b24110bbcf7004e01376ac&scope.system=all&role.id=78ec5dc2a67d445389f9a7a4458a12cb => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:48.394082 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe43e8-d77a-463c-9ed3-4b1a81e821b7 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:48.395057 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe43e8-d77a-463c-9ed3-4b1a81e821b7 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:48.395488 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe43e8-d77a-463c-9ed3-4b1a81e821b7 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:48.397759 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 181/366] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:48.398761 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe43e8-d77a-463c-9ed3-4b1a81e821b7 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:48.423941 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c508a8-5781-4717-91ed-a06b97c496ac None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:48.462263 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cdf0b-a1f4-4872-862a-6777d3ddc96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:48.463400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cdf0b-a1f4-4872-862a-6777d3ddc96f None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:48.463884 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cdf0b-a1f4-4872-862a-6777d3ddc96f None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:48.464305 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cdf0b-a1f4-4872-862a-6777d3ddc96f None admin] PATH_INFO: `/v3/groups/1ff36c94c8f6421da190b330cdbd9185` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:48.466115 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4cdf0b-a1f4-4872-862a-6777d3ddc96f None admin] RBAC: Authorizing `identity:get_group(group_id=1ff36c94c8f6421da190b330cdbd9185)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:48.467637 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4cdf0b-a1f4-4872-862a-6777d3ddc96f None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:48.469684 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 186/367] 10.222.0.26 () {56 vars in 1180 bytes} [Mon May 9 09:33:48 2022] GET /identity/v3/groups/1ff36c94c8f6421da190b330cdbd9185 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:48.470971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a2536-5ca7-4379-aada-54f8c46402cc None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:48.471253 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a2536-5ca7-4379-aada-54f8c46402cc None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:48.471522 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a2536-5ca7-4379-aada-54f8c46402cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:48.500880 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-326a2536-5ca7-4379-aada-54f8c46402cc None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:48.502019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-326a2536-5ca7-4379-aada-54f8c46402cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:48.542960 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe43e8-d77a-463c-9ed3-4b1a81e821b7 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:48.640495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bee54a-7d12-4b48-a310-cdc33f09307a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:48.642142 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bee54a-7d12-4b48-a310-cdc33f09307a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:48.642673 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bee54a-7d12-4b48-a310-cdc33f09307a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:48.643144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bee54a-7d12-4b48-a310-cdc33f09307a None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:48.644114 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bee54a-7d12-4b48-a310-cdc33f09307a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:48.646242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bee54a-7d12-4b48-a310-cdc33f09307a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:48.649664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bee54a-7d12-4b48-a310-cdc33f09307a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:48.666694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 187/368] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:48.668618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a2536-5ca7-4379-aada-54f8c46402cc None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:48.669796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 182/369] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:33:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:48.682053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bee54a-7d12-4b48-a310-cdc33f09307a None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:48.804011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a2536-5ca7-4379-aada-54f8c46402cc None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:48.814508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bee54a-7d12-4b48-a310-cdc33f09307a None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:48.844507 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810ac80-6f07-4103-9b52-187a615f3c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:48.845217 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810ac80-6f07-4103-9b52-187a615f3c50 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:48.845495 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810ac80-6f07-4103-9b52-187a615f3c50 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:48.845768 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810ac80-6f07-4103-9b52-187a615f3c50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:48.846616 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810ac80-6f07-4103-9b52-187a615f3c50 None admin] RBAC: Adding query filter params (group.id=1ff36c94c8f6421da190b330cdbd9185, role.id=0e3ae1b3fdce41c4a42a415efad3b84d, scope.project.id=9ec214e8e5724ce6b35fe31337cb0380) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:48.846930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810ac80-6f07-4103-9b52-187a615f3c50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:48.848429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810ac80-6f07-4103-9b52-187a615f3c50 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:48.854185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 188/370] 10.222.0.26 () {56 vars in 1376 bytes} [Mon May 9 09:33:48 2022] GET /identity/v3/role_assignments?group.id=1ff36c94c8f6421da190b330cdbd9185&scope.project.id=9ec214e8e5724ce6b35fe31337cb0380&role.id=0e3ae1b3fdce41c4a42a415efad3b84d => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:48.855171 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810ac80-6f07-4103-9b52-187a615f3c50 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:48.856376 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fe197-f465-4b62-affd-8cf16807d8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:48.857294 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fe197-f465-4b62-affd-8cf16807d8e6 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:48.857776 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fe197-f465-4b62-affd-8cf16807d8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:48.858249 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fe197-f465-4b62-affd-8cf16807d8e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:48.999220 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0fe197-f465-4b62-affd-8cf16807d8e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:49.002244 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0fe197-f465-4b62-affd-8cf16807d8e6 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:49.067696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810ac80-6f07-4103-9b52-187a615f3c50 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:49.113231 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2173f8-de3c-46ba-944e-798d936259df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:49.114039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2173f8-de3c-46ba-944e-798d936259df None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:49.114320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2173f8-de3c-46ba-944e-798d936259df None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:49.114592 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2173f8-de3c-46ba-944e-798d936259df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:49.125715 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2173f8-de3c-46ba-944e-798d936259df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:49.133497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-db2173f8-de3c-46ba-944e-798d936259df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:33:49.133497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:49.133497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:49.133497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:49.133497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:49.133497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:49.133497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:49.133497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:49.133497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:49.133497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:49.133497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:49.133497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:49.133497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:49.133497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:49.133497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:49.133497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:49.134413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:49.134413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:49.134413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:49.134413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:49.134413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:49.134413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:49.134413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:49.134413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:49.134413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:49.134413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:49.134413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:49.134413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:49.134413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:49.134413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:49.134413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:49.134413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:49.135202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:49.135202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:49.135202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:49.135202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:49.135202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:49.135202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:49.135202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:49.135202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:49.135202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:49.135202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:49.135202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:33:49.135202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:49.135202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 189/371] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:33:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:49.144947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2173f8-de3c-46ba-944e-798d936259df None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:49.168685 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 183/372] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 09:33:49.176248 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fe197-f465-4b62-affd-8cf16807d8e6 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:49.263227 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2173f8-de3c-46ba-944e-798d936259df None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:49.297920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01912e-0c67-4574-a889-ecf3591b7f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:49.298730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01912e-0c67-4574-a889-ecf3591b7f60 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:49.299070 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01912e-0c67-4574-a889-ecf3591b7f60 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:49.299402 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01912e-0c67-4574-a889-ecf3591b7f60 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:49.300016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fe197-f465-4b62-affd-8cf16807d8e6 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:49.300810 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d01912e-0c67-4574-a889-ecf3591b7f60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:49.301182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d01912e-0c67-4574-a889-ecf3591b7f60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:49.302684 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d01912e-0c67-4574-a889-ecf3591b7f60 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:49.315997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 190/373] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:49.317351 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01912e-0c67-4574-a889-ecf3591b7f60 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:49.345045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802be57-323e-4b61-8dda-1dcf47796dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:49.346059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802be57-323e-4b61-8dda-1dcf47796dac None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:49.346318 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802be57-323e-4b61-8dda-1dcf47796dac None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:49.346580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802be57-323e-4b61-8dda-1dcf47796dac None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:49.350452 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802be57-323e-4b61-8dda-1dcf47796dac None admin] RBAC: Authorizing `identity:get_project(project_id=9c3d1e7bc10449e2957923e1cbb5e09e)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:49.351931 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802be57-323e-4b61-8dda-1dcf47796dac None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:49.361431 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 184/374] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:49 2022] GET /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:49.373606 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802be57-323e-4b61-8dda-1dcf47796dac None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:49.495657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01912e-0c67-4574-a889-ecf3591b7f60 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:49.505791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802be57-323e-4b61-8dda-1dcf47796dac None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:49.569391 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7306d4-55bb-4808-9f0d-e9e732a727fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:49.570267 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7306d4-55bb-4808-9f0d-e9e732a727fb None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:49.570543 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7306d4-55bb-4808-9f0d-e9e732a727fb None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:49.570811 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7306d4-55bb-4808-9f0d-e9e732a727fb None admin] PATH_INFO: `/v3/groups/1ff36c94c8f6421da190b330cdbd9185` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:49.577485 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7306d4-55bb-4808-9f0d-e9e732a727fb None admin] RBAC: Authorizing `identity:get_group(group_id=1ff36c94c8f6421da190b330cdbd9185)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:49.580004 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7306d4-55bb-4808-9f0d-e9e732a727fb None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:49.581322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20769d5-4f5f-41da-8594-38a859236bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:49.581961 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20769d5-4f5f-41da-8594-38a859236bd3 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:49.582228 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20769d5-4f5f-41da-8594-38a859236bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:49.582489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20769d5-4f5f-41da-8594-38a859236bd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:49.584158 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 185/375] 10.222.0.26 () {56 vars in 1180 bytes} [Mon May 9 09:33:49 2022] GET /identity/v3/groups/1ff36c94c8f6421da190b330cdbd9185 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:49.590013 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7306d4-55bb-4808-9f0d-e9e732a727fb None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:49.710858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20769d5-4f5f-41da-8594-38a859236bd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:49.712252 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20769d5-4f5f-41da-8594-38a859236bd3 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:49.758026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7306d4-55bb-4808-9f0d-e9e732a727fb None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:49.795636 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55cbd80-eef2-4859-a23d-3cca7af263bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:49.796302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cbd80-eef2-4859-a23d-3cca7af263bd None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:49.796568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cbd80-eef2-4859-a23d-3cca7af263bd None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:49.796875 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cbd80-eef2-4859-a23d-3cca7af263bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:49.797710 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55cbd80-eef2-4859-a23d-3cca7af263bd None admin] RBAC: Adding query filter params (group.id=1ff36c94c8f6421da190b330cdbd9185, role.id=0e3ae1b3fdce41c4a42a415efad3b84d, scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:49.798019 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55cbd80-eef2-4859-a23d-3cca7af263bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:49.799461 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55cbd80-eef2-4859-a23d-3cca7af263bd None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:49.805943 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 186/376] 10.222.0.26 () {56 vars in 1376 bytes} [Mon May 9 09:33:49 2022] GET /identity/v3/role_assignments?group.id=1ff36c94c8f6421da190b330cdbd9185&scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e&role.id=0e3ae1b3fdce41c4a42a415efad3b84d => generated 260 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:49.807258 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9d2d7-8e15-42d2-a707-046e17d773ca None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:49.807545 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9d2d7-8e15-42d2-a707-046e17d773ca None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:49.807816 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9d2d7-8e15-42d2-a707-046e17d773ca None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:49.809331 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 187/377] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:49.815013 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3025324-5008-4383-b2e1-5c5c930e36f0 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:49.815349 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3025324-5008-4383-b2e1-5c5c930e36f0 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:49.815560 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3025324-5008-4383-b2e1-5c5c930e36f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:49.835628 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-f3025324-5008-4383-b2e1-5c5c930e36f0 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:49.836905 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f3025324-5008-4383-b2e1-5c5c930e36f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:49.844877 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 191/378] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:49.847019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20769d5-4f5f-41da-8594-38a859236bd3 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:49.910480 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 188/379] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:33:49.933133 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3025324-5008-4383-b2e1-5c5c930e36f0 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:49.986953 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20769d5-4f5f-41da-8594-38a859236bd3 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:50.037395 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00de18-4bd6-4879-92e4-8c625ff6b8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:50.038675 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00de18-4bd6-4879-92e4-8c625ff6b8da None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:50.039152 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00de18-4bd6-4879-92e4-8c625ff6b8da None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:50.039620 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00de18-4bd6-4879-92e4-8c625ff6b8da None admin] PATH_INFO: `/v3/projects/03d8c548fb514d70b477284ceb7b3252` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:50.041843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3025324-5008-4383-b2e1-5c5c930e36f0 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:50.042734 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00de18-4bd6-4879-92e4-8c625ff6b8da None admin] RBAC: Authorizing `identity:get_project(project_id=03d8c548fb514d70b477284ceb7b3252)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:50.044419 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00de18-4bd6-4879-92e4-8c625ff6b8da None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:50.046690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 192/380] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:49 2022] GET /identity/v3/projects/03d8c548fb514d70b477284ceb7b3252 => generated 344 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:50.051967 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00de18-4bd6-4879-92e4-8c625ff6b8da None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:50.076845 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bbcf5-280e-4647-aacc-18becfb73760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:50.078033 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bbcf5-280e-4647-aacc-18becfb73760 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:50.078540 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bbcf5-280e-4647-aacc-18becfb73760 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:50.079007 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bbcf5-280e-4647-aacc-18becfb73760 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:50.097356 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704bbcf5-280e-4647-aacc-18becfb73760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:50.106227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-704bbcf5-280e-4647-aacc-18becfb73760 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:50.106227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:50.106227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:50.106227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:50.106227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:50.106227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:50.106227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:50.106227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:50.106227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:50.106227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:50.106227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:50.106227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:50.106227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:50.106227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:50.106227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:50.106227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:50.107125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:50.107125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:50.107125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:50.107125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:50.107125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:50.107125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:50.107125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:50.107125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:50.107125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:50.107125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:50.107125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:50.107125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:33:50.107125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:50.107125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:50.107125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:50.107125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:50.107932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:50.107932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:50.107932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:50.107932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:50.107932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:50.107932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:50.107932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:50.107932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:50.107932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:50.107932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:50.107932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:50.107932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:33:50.121298 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 189/381] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:33:50.124042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bbcf5-280e-4647-aacc-18becfb73760 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:50.224094 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00de18-4bd6-4879-92e4-8c625ff6b8da None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:50.297439 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b009d-3507-4abd-bedb-728bc4b1cd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:50.298701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b009d-3507-4abd-bedb-728bc4b1cd9e None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:50.299200 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b009d-3507-4abd-bedb-728bc4b1cd9e None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:50.299668 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b009d-3507-4abd-bedb-728bc4b1cd9e None admin] PATH_INFO: `/v3/groups/e9a1c2c9bfb1473bb2947e144ceafa64` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:50.312522 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bbcf5-280e-4647-aacc-18becfb73760 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:50.313482 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b009d-3507-4abd-bedb-728bc4b1cd9e None admin] RBAC: Authorizing `identity:get_group(group_id=e9a1c2c9bfb1473bb2947e144ceafa64)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:50.315177 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b009d-3507-4abd-bedb-728bc4b1cd9e None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:50.319698 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 193/382] 10.222.0.26 () {56 vars in 1180 bytes} [Mon May 9 09:33:50 2022] GET /identity/v3/groups/e9a1c2c9bfb1473bb2947e144ceafa64 => generated 229 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:50.325553 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b009d-3507-4abd-bedb-728bc4b1cd9e None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:50.442060 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1aaf45-fdc2-4ecc-b719-9b9b838b0c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:50.442753 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1aaf45-fdc2-4ecc-b719-9b9b838b0c91 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:50.443315 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1aaf45-fdc2-4ecc-b719-9b9b838b0c91 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:50.443315 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1aaf45-fdc2-4ecc-b719-9b9b838b0c91 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:50.444025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1aaf45-fdc2-4ecc-b719-9b9b838b0c91 None admin] RBAC: Adding query filter params (name=member) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:50.444327 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1aaf45-fdc2-4ecc-b719-9b9b838b0c91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:50.445763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1aaf45-fdc2-4ecc-b719-9b9b838b0c91 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:50.456959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b009d-3507-4abd-bedb-728bc4b1cd9e None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:50.479422 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 190/383] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:50.489765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1aaf45-fdc2-4ecc-b719-9b9b838b0c91 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:50.503070 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b269b0-3a28-49fd-91b1-ef3899807206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:50.504112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b269b0-3a28-49fd-91b1-ef3899807206 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:50.504606 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b269b0-3a28-49fd-91b1-ef3899807206 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:50.505114 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b269b0-3a28-49fd-91b1-ef3899807206 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:50.506171 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b269b0-3a28-49fd-91b1-ef3899807206 None admin] RBAC: Adding query filter params (group.id=e9a1c2c9bfb1473bb2947e144ceafa64, role.id=70ac549382b14b2cbb8b35e5dfc648d4, scope.project.id=03d8c548fb514d70b477284ceb7b3252) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:50.506676 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b269b0-3a28-49fd-91b1-ef3899807206 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:50.508322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b269b0-3a28-49fd-91b1-ef3899807206 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:50.524552 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 194/384] 10.222.0.26 () {56 vars in 1376 bytes} [Mon May 9 09:33:50 2022] GET /identity/v3/role_assignments?group.id=e9a1c2c9bfb1473bb2947e144ceafa64&scope.project.id=03d8c548fb514d70b477284ceb7b3252&role.id=70ac549382b14b2cbb8b35e5dfc648d4 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:50.688703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1aaf45-fdc2-4ecc-b719-9b9b838b0c91 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:50.751249 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ba8b7-732a-48f9-bfd1-dabe32744b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:50.752122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ba8b7-732a-48f9-bfd1-dabe32744b63 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:50.752390 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ba8b7-732a-48f9-bfd1-dabe32744b63 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:50.753013 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ba8b7-732a-48f9-bfd1-dabe32744b63 None admin] PATH_INFO: `/v3/groups/1ff36c94c8f6421da190b330cdbd9185` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:50.753850 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ba8b7-732a-48f9-bfd1-dabe32744b63 None admin] RBAC: Authorizing `identity:get_group(group_id=1ff36c94c8f6421da190b330cdbd9185)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:50.755376 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ba8b7-732a-48f9-bfd1-dabe32744b63 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:50.757337 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 191/385] 10.222.0.26 () {56 vars in 1180 bytes} [Mon May 9 09:33:50 2022] GET /identity/v3/groups/1ff36c94c8f6421da190b330cdbd9185 => generated 226 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:50.762855 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b269b0-3a28-49fd-91b1-ef3899807206 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:50.907967 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b269b0-3a28-49fd-91b1-ef3899807206 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:50.944234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4714b76-5dbb-4bf3-887b-2f1cd49cf9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:50.945170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4714b76-5dbb-4bf3-887b-2f1cd49cf9a4 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:50.945447 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4714b76-5dbb-4bf3-887b-2f1cd49cf9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:50.945713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4714b76-5dbb-4bf3-887b-2f1cd49cf9a4 None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:50.947257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4714b76-5dbb-4bf3-887b-2f1cd49cf9a4 None admin] RBAC: Authorizing `identity:get_project(project_id=9ec214e8e5724ce6b35fe31337cb0380)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:50.949349 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4714b76-5dbb-4bf3-887b-2f1cd49cf9a4 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:50.957364 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 195/386] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:50 2022] GET /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:50.959850 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ba8b7-732a-48f9-bfd1-dabe32744b63 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:51.096301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ba8b7-732a-48f9-bfd1-dabe32744b63 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:51.131460 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9da84c-33f4-4144-a2cd-37cac49c5ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:51.132378 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9da84c-33f4-4144-a2cd-37cac49c5ab0 None admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:51.132646 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9da84c-33f4-4144-a2cd-37cac49c5ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:51.132946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9da84c-33f4-4144-a2cd-37cac49c5ab0 None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380/groups/1ff36c94c8f6421da190b330cdbd9185/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:51.134927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9da84c-33f4-4144-a2cd-37cac49c5ab0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ec214e8e5724ce6b35fe31337cb0380, group_id=1ff36c94c8f6421da190b330cdbd9185, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:51.136466 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9da84c-33f4-4144-a2cd-37cac49c5ab0 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:51.159103 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 192/387] 10.222.0.26 () {58 vars in 1442 bytes} [Mon May 9 09:33:50 2022] PUT /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380/groups/1ff36c94c8f6421da190b330cdbd9185/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:33:51.180871 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58273cd-5062-474b-a7d7-5684d99395b4 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:51.181194 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58273cd-5062-474b-a7d7-5684d99395b4 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:51.181426 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58273cd-5062-474b-a7d7-5684d99395b4 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:51.182977 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 196/388] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:51.189032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df642f2-bcfa-409d-8abe-8de93954df61 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:51.189318 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df642f2-bcfa-409d-8abe-8de93954df61 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:51.189640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df642f2-bcfa-409d-8abe-8de93954df61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:51.210444 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-3df642f2-bcfa-409d-8abe-8de93954df61 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:51.211662 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-3df642f2-bcfa-409d-8abe-8de93954df61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:51.289764 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 193/389] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:51.315011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58273cd-5062-474b-a7d7-5684d99395b4 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:51.433908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58273cd-5062-474b-a7d7-5684d99395b4 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:51.467902 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85242e2-c60e-4506-af71-9db35cce4e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:51.468768 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85242e2-c60e-4506-af71-9db35cce4e2b None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:51.469045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85242e2-c60e-4506-af71-9db35cce4e2b None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:51.469312 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85242e2-c60e-4506-af71-9db35cce4e2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:51.479738 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85242e2-c60e-4506-af71-9db35cce4e2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:51.487671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-e85242e2-c60e-4506-af71-9db35cce4e2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:51.487671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:51.487671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:51.487671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:51.487671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:51.487671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:51.487671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:51.487671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:51.487671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:51.487671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:51.487671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:51.487671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:51.487671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:51.487671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:51.487671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:51.487671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:51.488556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:51.488556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:51.488556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:51.488556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:51.488556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:51.488556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:51.488556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:51.488556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:51.488556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:51.488556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:51.488556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:51.488556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:51.488556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:51.488556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:51.488556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:51.488556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:51.489407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:51.489407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:51.489407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:51.489407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:51.489407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:51.489407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:51.489407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:51.489407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:51.489407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:51.489407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:51.489407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:51.489407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:51.489407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 197/390] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:51.494806 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df642f2-bcfa-409d-8abe-8de93954df61 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:51.599251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df642f2-bcfa-409d-8abe-8de93954df61 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:51.633513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355eae7f-f2bc-4bfe-aec6-e433744058c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:51.634249 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355eae7f-f2bc-4bfe-aec6-e433744058c7 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:51.634524 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355eae7f-f2bc-4bfe-aec6-e433744058c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:51.634788 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355eae7f-f2bc-4bfe-aec6-e433744058c7 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:51.635622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355eae7f-f2bc-4bfe-aec6-e433744058c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:51.635925 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355eae7f-f2bc-4bfe-aec6-e433744058c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:51.637325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355eae7f-f2bc-4bfe-aec6-e433744058c7 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:51.646529 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 194/391] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:51.653991 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85242e2-c60e-4506-af71-9db35cce4e2b None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:51.780235 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85242e2-c60e-4506-af71-9db35cce4e2b None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:51.813413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f3623-9703-4596-be5c-fe57b5920638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:51.814282 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f3623-9703-4596-be5c-fe57b5920638 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:51.814556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f3623-9703-4596-be5c-fe57b5920638 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:51.814823 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f3623-9703-4596-be5c-fe57b5920638 None admin] PATH_INFO: `/v3/groups/1ff36c94c8f6421da190b330cdbd9185` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:51.816115 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f3623-9703-4596-be5c-fe57b5920638 None admin] RBAC: Authorizing `identity:get_group(group_id=1ff36c94c8f6421da190b330cdbd9185)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:51.817600 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f3623-9703-4596-be5c-fe57b5920638 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:51.819551 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 198/392] 10.222.0.26 () {56 vars in 1180 bytes} [Mon May 9 09:33:51 2022] GET /identity/v3/groups/1ff36c94c8f6421da190b330cdbd9185 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:51.824864 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355eae7f-f2bc-4bfe-aec6-e433744058c7 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:51.892360 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e7728-ec4b-4cfb-bc8c-14bf13f2b8cc None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:51.892718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e7728-ec4b-4cfb-bc8c-14bf13f2b8cc None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:51.892916 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e7728-ec4b-4cfb-bc8c-14bf13f2b8cc None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:51.894290 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 199/393] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:33:51.902496 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7677736-069e-4965-aaea-d850a83a40bd None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:51.902771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7677736-069e-4965-aaea-d850a83a40bd None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:51.903016 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7677736-069e-4965-aaea-d850a83a40bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:51.920723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-b7677736-069e-4965-aaea-d850a83a40bd None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:51.921891 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b7677736-069e-4965-aaea-d850a83a40bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:51.954838 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355eae7f-f2bc-4bfe-aec6-e433744058c7 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:51.996319 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8cfd22-bc8d-402f-ac98-77f738011926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:51.997176 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cfd22-bc8d-402f-ac98-77f738011926 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:51.997420 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cfd22-bc8d-402f-ac98-77f738011926 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:51.997688 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cfd22-bc8d-402f-ac98-77f738011926 None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:52.000003 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8cfd22-bc8d-402f-ac98-77f738011926 None admin] RBAC: Authorizing `identity:get_project(project_id=9c3d1e7bc10449e2957923e1cbb5e09e)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:52.003565 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8cfd22-bc8d-402f-ac98-77f738011926 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:52.006581 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 195/394] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:51 2022] GET /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:52.006933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 200/395] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:33:52.012360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8cfd22-bc8d-402f-ac98-77f738011926 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:52.032411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7677736-069e-4965-aaea-d850a83a40bd None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:52.132923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7677736-069e-4965-aaea-d850a83a40bd None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:52.169691 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e21cd-9581-4920-90d8-516ce476dfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:52.170502 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e21cd-9581-4920-90d8-516ce476dfdf None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:52.170776 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e21cd-9581-4920-90d8-516ce476dfdf None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:52.171040 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e21cd-9581-4920-90d8-516ce476dfdf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:52.194792 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e21cd-9581-4920-90d8-516ce476dfdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:52.197987 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8cfd22-bc8d-402f-ac98-77f738011926 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:52.206218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-0c8e21cd-9581-4920-90d8-516ce476dfdf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:52.206218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:52.206218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:52.206218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:52.206218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:52.206218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:52.206218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:52.206218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:52.206218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:52.206218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:52.206218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:52.206218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:52.206218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:52.206218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:52.206218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:52.206218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:52.207107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:52.207107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:52.207107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:52.207107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:52.207107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:52.207107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:52.207107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:52.207107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:52.207107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:52.207107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:52.207107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:52.207107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:52.207107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:52.207107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:52.207107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:52.207107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:52.207923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:52.207923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:52.207923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:52.207923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:52.207923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:52.207923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:52.207923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:52.207923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:52.207923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:52.207923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:52.207923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:52.207923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:52.211719 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 201/396] 10.222.0.26 () {56 vars in 1114 bytes} [Mon May 9 09:33:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:52.218212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e21cd-9581-4920-90d8-516ce476dfdf None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:52.272087 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ca058-df5b-4047-a9d9-3b83c009f4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:52.273367 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ca058-df5b-4047-a9d9-3b83c009f4ac None admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:52.273865 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ca058-df5b-4047-a9d9-3b83c009f4ac None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:52.274351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ca058-df5b-4047-a9d9-3b83c009f4ac None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e/groups/1ff36c94c8f6421da190b330cdbd9185/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:52.278588 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ca058-df5b-4047-a9d9-3b83c009f4ac None admin] RBAC: Authorizing `identity:create_grant(project_id=9c3d1e7bc10449e2957923e1cbb5e09e, group_id=1ff36c94c8f6421da190b330cdbd9185, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:52.281001 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ca058-df5b-4047-a9d9-3b83c009f4ac None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:52.303806 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 196/397] 10.222.0.26 () {58 vars in 1442 bytes} [Mon May 9 09:33:52 2022] PUT /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e/groups/1ff36c94c8f6421da190b330cdbd9185/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:33:52.375803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e21cd-9581-4920-90d8-516ce476dfdf None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:52.417332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6c891-5fac-4d0b-a474-bdd6a924d0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:52.418078 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6c891-5fac-4d0b-a474-bdd6a924d0b0 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:52.418358 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6c891-5fac-4d0b-a474-bdd6a924d0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:52.418626 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6c891-5fac-4d0b-a474-bdd6a924d0b0 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:52.419391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b6c891-5fac-4d0b-a474-bdd6a924d0b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:52.419689 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b6c891-5fac-4d0b-a474-bdd6a924d0b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:52.421130 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b6c891-5fac-4d0b-a474-bdd6a924d0b0 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:52.434788 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 202/398] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:33:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:52.440633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ca058-df5b-4047-a9d9-3b83c009f4ac None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:52.571904 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ca058-df5b-4047-a9d9-3b83c009f4ac None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:52.614325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26130e63-7b4d-4c3a-b81d-16f086d53b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:52.615611 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26130e63-7b4d-4c3a-b81d-16f086d53b85 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:52.616122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26130e63-7b4d-4c3a-b81d-16f086d53b85 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:52.616594 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26130e63-7b4d-4c3a-b81d-16f086d53b85 None admin] PATH_INFO: `/v3/groups/1ff36c94c8f6421da190b330cdbd9185` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:52.618183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26130e63-7b4d-4c3a-b81d-16f086d53b85 None admin] RBAC: Authorizing `identity:get_group(group_id=1ff36c94c8f6421da190b330cdbd9185)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:52.619941 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26130e63-7b4d-4c3a-b81d-16f086d53b85 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:52.622622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 197/399] 10.222.0.26 () {56 vars in 1180 bytes} [Mon May 9 09:33:52 2022] GET /identity/v3/groups/1ff36c94c8f6421da190b330cdbd9185 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:52.628525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6c891-5fac-4d0b-a474-bdd6a924d0b0 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:52.741391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6c891-5fac-4d0b-a474-bdd6a924d0b0 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:52.779206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029a28d-9ab1-4747-b312-e3fd04a471b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:52.780081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029a28d-9ab1-4747-b312-e3fd04a471b1 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:52.780356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029a28d-9ab1-4747-b312-e3fd04a471b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:52.780933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029a28d-9ab1-4747-b312-e3fd04a471b1 None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:52.781835 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029a28d-9ab1-4747-b312-e3fd04a471b1 None admin] RBAC: Authorizing `identity:get_project(project_id=9ec214e8e5724ce6b35fe31337cb0380)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:52.783281 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029a28d-9ab1-4747-b312-e3fd04a471b1 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:52.785175 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 203/400] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:52 2022] GET /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:52.791042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26130e63-7b4d-4c3a-b81d-16f086d53b85 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:52.885395 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80777f-082f-4e9e-986c-293f44fe8f04 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:52.886622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80777f-082f-4e9e-986c-293f44fe8f04 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:52.886733 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80777f-082f-4e9e-986c-293f44fe8f04 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:52.888112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 204/401] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:33:52.894292 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c05ee-b1b4-479d-920b-0f559c13171d None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:52.894572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c05ee-b1b4-479d-920b-0f559c13171d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:52.894848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c05ee-b1b4-479d-920b-0f559c13171d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:52.923473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-c29c05ee-b1b4-479d-920b-0f559c13171d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:52.924701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c29c05ee-b1b4-479d-920b-0f559c13171d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:52.960641 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26130e63-7b4d-4c3a-b81d-16f086d53b85 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:53.007694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcda5b-dbf3-4a2e-a31c-bb52d9698844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:53.008579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcda5b-dbf3-4a2e-a31c-bb52d9698844 None admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:53.008878 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcda5b-dbf3-4a2e-a31c-bb52d9698844 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:53.009149 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcda5b-dbf3-4a2e-a31c-bb52d9698844 None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380/groups/1ff36c94c8f6421da190b330cdbd9185/roles/0e3ae1b3fdce41c4a42a415efad3b84d` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:53.012737 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dcda5b-dbf3-4a2e-a31c-bb52d9698844 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ec214e8e5724ce6b35fe31337cb0380, group_id=1ff36c94c8f6421da190b330cdbd9185, role_id=0e3ae1b3fdce41c4a42a415efad3b84d)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:53.014247 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dcda5b-dbf3-4a2e-a31c-bb52d9698844 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:53.023319 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 205/402] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:33:53.038415 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 198/403] 10.222.0.26 () {58 vars in 1442 bytes} [Mon May 9 09:33:52 2022] PUT /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380/groups/1ff36c94c8f6421da190b330cdbd9185/roles/0e3ae1b3fdce41c4a42a415efad3b84d => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:33:53.045930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c05ee-b1b4-479d-920b-0f559c13171d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:53.154698 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c05ee-b1b4-479d-920b-0f559c13171d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:53.194470 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f0700-946e-4533-8880-6d689785e38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:53.195708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f0700-946e-4533-8880-6d689785e38f None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:53.196188 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f0700-946e-4533-8880-6d689785e38f None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:53.196652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f0700-946e-4533-8880-6d689785e38f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:53.212920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f0700-946e-4533-8880-6d689785e38f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:53.221846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-7b3f0700-946e-4533-8880-6d689785e38f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:53.221846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:53.221846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:53.221846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:53.221846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:53.221846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:53.221846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:53.221846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:53.221846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:53.221846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:53.221846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:53.221846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:53.221846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:53.221846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:53.221846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:53.221846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:53.222737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:53.222737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:53.222737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:53.222737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:53.222737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:53.222737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:53.222737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:53.222737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:53.222737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:53.222737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:53.222737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:53.222737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:53.222737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:53.222737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:53.222737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:53.222737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:53.223604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:53.223604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:53.223604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:53.223604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:53.223604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:53.223604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:53.223604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:53.223604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:53.223604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:53.223604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:53.223604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:53.223604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:53.228688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 206/404] 10.222.0.26 () {56 vars in 1114 bytes} [Mon May 9 09:33:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:53.233841 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcda5b-dbf3-4a2e-a31c-bb52d9698844 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:53.358472 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcda5b-dbf3-4a2e-a31c-bb52d9698844 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:53.392344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102cd153-3c08-4fc7-8f81-3b7e68ae869e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:53.393389 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102cd153-3c08-4fc7-8f81-3b7e68ae869e None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:53.393873 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102cd153-3c08-4fc7-8f81-3b7e68ae869e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:53.394343 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102cd153-3c08-4fc7-8f81-3b7e68ae869e None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:53.395292 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102cd153-3c08-4fc7-8f81-3b7e68ae869e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:53.396020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102cd153-3c08-4fc7-8f81-3b7e68ae869e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:53.397625 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102cd153-3c08-4fc7-8f81-3b7e68ae869e None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:53.408361 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 199/405] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:33:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:53.413688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f0700-946e-4533-8880-6d689785e38f None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:53.522106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f0700-946e-4533-8880-6d689785e38f None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:53.569919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99977ccf-24d3-4b7a-b169-f0c7d32084a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:53.571116 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99977ccf-24d3-4b7a-b169-f0c7d32084a1 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:53.571592 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99977ccf-24d3-4b7a-b169-f0c7d32084a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:53.572059 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99977ccf-24d3-4b7a-b169-f0c7d32084a1 None admin] PATH_INFO: `/v3/groups/1ff36c94c8f6421da190b330cdbd9185` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:53.573571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99977ccf-24d3-4b7a-b169-f0c7d32084a1 None admin] RBAC: Authorizing `identity:get_group(group_id=1ff36c94c8f6421da190b330cdbd9185)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:53.575210 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99977ccf-24d3-4b7a-b169-f0c7d32084a1 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:53.577403 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 207/406] 10.222.0.26 () {56 vars in 1180 bytes} [Mon May 9 09:33:53 2022] GET /identity/v3/groups/1ff36c94c8f6421da190b330cdbd9185 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:53.582837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102cd153-3c08-4fc7-8f81-3b7e68ae869e None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:53.704336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102cd153-3c08-4fc7-8f81-3b7e68ae869e None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:53.750579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d3265-4dd8-4e9f-bdb8-da76991d8655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:53.751816 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d3265-4dd8-4e9f-bdb8-da76991d8655 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:53.752298 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d3265-4dd8-4e9f-bdb8-da76991d8655 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:53.752801 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d3265-4dd8-4e9f-bdb8-da76991d8655 None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:53.754199 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d3265-4dd8-4e9f-bdb8-da76991d8655 None admin] RBAC: Authorizing `identity:get_project(project_id=9c3d1e7bc10449e2957923e1cbb5e09e)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:53.755890 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d3265-4dd8-4e9f-bdb8-da76991d8655 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:53.758148 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 200/407] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:53 2022] GET /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:53.764000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99977ccf-24d3-4b7a-b169-f0c7d32084a1 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:53.891707 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99977ccf-24d3-4b7a-b169-f0c7d32084a1 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:53.941066 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dab172-7bc6-4a49-b36a-78224109da45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:53.941993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dab172-7bc6-4a49-b36a-78224109da45 None admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:53.942267 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dab172-7bc6-4a49-b36a-78224109da45 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:53.942538 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dab172-7bc6-4a49-b36a-78224109da45 None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e/groups/1ff36c94c8f6421da190b330cdbd9185/roles/0e3ae1b3fdce41c4a42a415efad3b84d` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:53.948163 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4250956-dce2-4677-a81a-03cab99e08ce None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:53.948443 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4250956-dce2-4677-a81a-03cab99e08ce None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:53.948741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4250956-dce2-4677-a81a-03cab99e08ce None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:53.950462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 201/408] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:53.952244 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dab172-7bc6-4a49-b36a-78224109da45 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c3d1e7bc10449e2957923e1cbb5e09e, group_id=1ff36c94c8f6421da190b330cdbd9185, role_id=0e3ae1b3fdce41c4a42a415efad3b84d)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:53.953745 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dab172-7bc6-4a49-b36a-78224109da45 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:53.955599 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0690e6-123a-4528-97e0-3a0169b1a74d None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:53.955813 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0690e6-123a-4528-97e0-3a0169b1a74d None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:53.956079 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0690e6-123a-4528-97e0-3a0169b1a74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:53.977602 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-ea0690e6-123a-4528-97e0-3a0169b1a74d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:53.979453 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 208/409] 10.222.0.26 () {58 vars in 1442 bytes} [Mon May 9 09:33:53 2022] PUT /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e/groups/1ff36c94c8f6421da190b330cdbd9185/roles/0e3ae1b3fdce41c4a42a415efad3b84d => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:33:53.985812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-ea0690e6-123a-4528-97e0-3a0169b1a74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:54.075284 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 202/410] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:33:54.108812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dab172-7bc6-4a49-b36a-78224109da45 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:54.249341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dab172-7bc6-4a49-b36a-78224109da45 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:54.289554 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466c07d-af86-4ced-add7-0c482524b366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:54.290357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466c07d-af86-4ced-add7-0c482524b366 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:54.290637 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466c07d-af86-4ced-add7-0c482524b366 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:54.290923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466c07d-af86-4ced-add7-0c482524b366 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:54.302547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c466c07d-af86-4ced-add7-0c482524b366 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:54.310280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-c466c07d-af86-4ced-add7-0c482524b366 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:33:54.310280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:54.310280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:54.310280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:54.310280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:54.310280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:54.310280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:54.310280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:54.310280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:54.310280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:54.310280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:54.310280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:54.310280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:54.310280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:54.310280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:54.310280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:54.311191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:54.311191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:54.311191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:54.311191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:54.311191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:54.311191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:54.311191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:54.311191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:54.311191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:54.311191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:54.311191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:54.311191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:54.311191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:54.311191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:54.311191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:54.311191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:54.312051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:54.312051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:54.312051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:54.312051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:54.312051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:54.312051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:54.312051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:54.312051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:54.312051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:54.312051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:54.312051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:33:54.312051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:54.312051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 209/411] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:33:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:54.317413 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0690e6-123a-4528-97e0-3a0169b1a74d None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:54.409593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46020fa-49e5-4b32-95e3-81273f513797 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:54.409916 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46020fa-49e5-4b32-95e3-81273f513797 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:54.410140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46020fa-49e5-4b32-95e3-81273f513797 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:54.411754 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 210/412] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:33:54.416764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaefa5e-a9e5-4a0d-9499-2e82e4906e33 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:54.417036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaefa5e-a9e5-4a0d-9499-2e82e4906e33 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:54.417313 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaefa5e-a9e5-4a0d-9499-2e82e4906e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:54.444554 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-8eaefa5e-a9e5-4a0d-9499-2e82e4906e33 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:54.445826 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-8eaefa5e-a9e5-4a0d-9499-2e82e4906e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:54.482547 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0690e6-123a-4528-97e0-3a0169b1a74d None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:54.517724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bd31a-1288-4386-8c21-daf5c48eefdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:54.518301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bd31a-1288-4386-8c21-daf5c48eefdc None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:54.518574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bd31a-1288-4386-8c21-daf5c48eefdc None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:54.518841 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bd31a-1288-4386-8c21-daf5c48eefdc None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:54.519754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3bd31a-1288-4386-8c21-daf5c48eefdc None admin] RBAC: Adding query filter params (name=admin) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:54.520059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3bd31a-1288-4386-8c21-daf5c48eefdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:54.521462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3bd31a-1288-4386-8c21-daf5c48eefdc None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:54.531200 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 203/413] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:54.536514 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bd31a-1288-4386-8c21-daf5c48eefdc None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:54.583719 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 211/414] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:33:54.648605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaefa5e-a9e5-4a0d-9499-2e82e4906e33 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:54.670642 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bd31a-1288-4386-8c21-daf5c48eefdc None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:54.741327 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c6509-8b66-4f8d-9648-bde308d85301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:54.742193 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c6509-8b66-4f8d-9648-bde308d85301 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:54.742469 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c6509-8b66-4f8d-9648-bde308d85301 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:54.742738 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c6509-8b66-4f8d-9648-bde308d85301 None admin] PATH_INFO: `/v3/groups/e9a1c2c9bfb1473bb2947e144ceafa64` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:54.749434 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c6509-8b66-4f8d-9648-bde308d85301 None admin] RBAC: Authorizing `identity:get_group(group_id=e9a1c2c9bfb1473bb2947e144ceafa64)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:54.750963 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c6509-8b66-4f8d-9648-bde308d85301 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:54.752991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 204/415] 10.222.0.26 () {56 vars in 1180 bytes} [Mon May 9 09:33:54 2022] GET /identity/v3/groups/e9a1c2c9bfb1473bb2947e144ceafa64 => generated 229 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:54.758270 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c6509-8b66-4f8d-9648-bde308d85301 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:54.792440 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaefa5e-a9e5-4a0d-9499-2e82e4906e33 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:54.829478 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59c0a5-46b3-4bf4-991a-8580519a1bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:54.830274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59c0a5-46b3-4bf4-991a-8580519a1bf1 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:54.830547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59c0a5-46b3-4bf4-991a-8580519a1bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:54.830812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59c0a5-46b3-4bf4-991a-8580519a1bf1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:54.841089 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59c0a5-46b3-4bf4-991a-8580519a1bf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:54.850590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-1e59c0a5-46b3-4bf4-991a-8580519a1bf1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:54.850590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:54.850590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:54.850590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:54.850590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:54.850590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:54.850590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:54.850590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:54.850590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:54.850590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:54.850590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:54.850590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:54.850590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:54.850590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:54.850590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:54.850590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:54.851525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:54.851525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:54.851525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:54.851525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:54.851525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:54.851525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:54.851525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:54.851525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:54.851525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:54.851525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:54.851525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:54.851525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:54.851525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:54.851525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:54.851525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:54.851525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:54.852346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:54.852346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:54.852346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:54.852346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:54.852346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:54.852346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:54.852346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:54.852346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:54.852346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:54.852346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:54.852346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:54.852346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:54.852346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 212/416] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:54.858544 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59c0a5-46b3-4bf4-991a-8580519a1bf1 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:54.892400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c6509-8b66-4f8d-9648-bde308d85301 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:54.949397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c08916-4876-4c65-91ef-d9f735dfb01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:54.950277 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c08916-4876-4c65-91ef-d9f735dfb01d None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:54.950557 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c08916-4876-4c65-91ef-d9f735dfb01d None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:54.950828 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c08916-4876-4c65-91ef-d9f735dfb01d None admin] PATH_INFO: `/v3/projects/03d8c548fb514d70b477284ceb7b3252` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:54.952058 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c08916-4876-4c65-91ef-d9f735dfb01d None admin] RBAC: Authorizing `identity:get_project(project_id=03d8c548fb514d70b477284ceb7b3252)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:54.953544 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c08916-4876-4c65-91ef-d9f735dfb01d None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:54.955561 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 205/417] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:54 2022] GET /identity/v3/projects/03d8c548fb514d70b477284ceb7b3252 => generated 344 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:54.956156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59c0a5-46b3-4bf4-991a-8580519a1bf1 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:54.960939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c08916-4876-4c65-91ef-d9f735dfb01d None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:54.990230 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918b49c-e39b-44ee-b834-50a360121f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:54.990947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918b49c-e39b-44ee-b834-50a360121f16 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:54.991219 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918b49c-e39b-44ee-b834-50a360121f16 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:54.991532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918b49c-e39b-44ee-b834-50a360121f16 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:54.992279 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918b49c-e39b-44ee-b834-50a360121f16 None admin] RBAC: Adding query filter params (name=member) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:54.992576 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918b49c-e39b-44ee-b834-50a360121f16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:54.994046 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918b49c-e39b-44ee-b834-50a360121f16 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:55.003446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 213/418] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:55.008918 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918b49c-e39b-44ee-b834-50a360121f16 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:55.092261 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c08916-4876-4c65-91ef-d9f735dfb01d None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:55.106912 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918b49c-e39b-44ee-b834-50a360121f16 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:55.133811 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1bd15-a9ae-4063-9fa6-18a744e8b3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:55.134720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1bd15-a9ae-4063-9fa6-18a744e8b3b8 None admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:55.134998 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1bd15-a9ae-4063-9fa6-18a744e8b3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:55.135350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1bd15-a9ae-4063-9fa6-18a744e8b3b8 None admin] PATH_INFO: `/v3/projects/03d8c548fb514d70b477284ceb7b3252/groups/e9a1c2c9bfb1473bb2947e144ceafa64/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:55.140951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d2bda-3393-4ccc-b614-546591f240d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:55.141609 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d2bda-3393-4ccc-b614-546591f240d0 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:55.141917 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d2bda-3393-4ccc-b614-546591f240d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:55.142134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d2bda-3393-4ccc-b614-546591f240d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:55.142313 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1bd15-a9ae-4063-9fa6-18a744e8b3b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=03d8c548fb514d70b477284ceb7b3252, group_id=e9a1c2c9bfb1473bb2947e144ceafa64, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:55.143668 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1bd15-a9ae-4063-9fa6-18a744e8b3b8 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:55.170790 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 206/419] 10.222.0.26 () {58 vars in 1442 bytes} [Mon May 9 09:33:54 2022] PUT /identity/v3/projects/03d8c548fb514d70b477284ceb7b3252/groups/e9a1c2c9bfb1473bb2947e144ceafa64/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:33:55.216913 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1d2bda-3393-4ccc-b614-546591f240d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:55.218411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1d2bda-3393-4ccc-b614-546591f240d0 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:55.323430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 214/420] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:55.329117 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1bd15-a9ae-4063-9fa6-18a744e8b3b8 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:55.368265 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367612fb-61e2-4426-a2e4-206b64f27425 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:55.368595 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367612fb-61e2-4426-a2e4-206b64f27425 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:55.368782 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367612fb-61e2-4426-a2e4-206b64f27425 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:55.370237 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 215/421] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:33:55.386412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2166bf-79c7-4fa2-bcb3-436fc1bcee78 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:55.386755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2166bf-79c7-4fa2-bcb3-436fc1bcee78 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:55.386916 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2166bf-79c7-4fa2-bcb3-436fc1bcee78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:55.415848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-5e2166bf-79c7-4fa2-bcb3-436fc1bcee78 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:55.417143 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-5e2166bf-79c7-4fa2-bcb3-436fc1bcee78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:55.453893 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1bd15-a9ae-4063-9fa6-18a744e8b3b8 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:55.499321 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619037e-eefa-4fcd-9b4c-c6cbc29b7c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:55.500185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619037e-eefa-4fcd-9b4c-c6cbc29b7c7f None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:55.500459 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619037e-eefa-4fcd-9b4c-c6cbc29b7c7f None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:55.500750 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619037e-eefa-4fcd-9b4c-c6cbc29b7c7f None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:55.501999 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619037e-eefa-4fcd-9b4c-c6cbc29b7c7f None admin] RBAC: Authorizing `identity:get_project(project_id=9ec214e8e5724ce6b35fe31337cb0380)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:55.503481 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619037e-eefa-4fcd-9b4c-c6cbc29b7c7f None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:55.505551 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 207/422] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:55 2022] GET /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:55.510867 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619037e-eefa-4fcd-9b4c-c6cbc29b7c7f None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:55.521983 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 216/423] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:33:55.555014 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2166bf-79c7-4fa2-bcb3-436fc1bcee78 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:55.630489 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619037e-eefa-4fcd-9b4c-c6cbc29b7c7f None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:55.670110 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f01a77-d93d-44cd-8954-d056c839080f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:55.671322 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f01a77-d93d-44cd-8954-d056c839080f None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:55.671807 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f01a77-d93d-44cd-8954-d056c839080f None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:55.672402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f01a77-d93d-44cd-8954-d056c839080f None admin] PATH_INFO: `/v3/groups/1ff36c94c8f6421da190b330cdbd9185` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:55.674508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f01a77-d93d-44cd-8954-d056c839080f None admin] RBAC: Authorizing `identity:get_group(group_id=1ff36c94c8f6421da190b330cdbd9185)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:55.676196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f01a77-d93d-44cd-8954-d056c839080f None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:55.678824 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 208/424] 10.222.0.26 () {56 vars in 1180 bytes} [Mon May 9 09:33:55 2022] GET /identity/v3/groups/1ff36c94c8f6421da190b330cdbd9185 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:55.684609 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f01a77-d93d-44cd-8954-d056c839080f None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:55.774693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2166bf-79c7-4fa2-bcb3-436fc1bcee78 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:55.800585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f01a77-d93d-44cd-8954-d056c839080f None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:55.837998 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277be1f-d0e7-478d-bdca-107dc8b636d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:55.839295 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277be1f-d0e7-478d-bdca-107dc8b636d2 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:55.839862 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277be1f-d0e7-478d-bdca-107dc8b636d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:55.840410 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277be1f-d0e7-478d-bdca-107dc8b636d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:55.841419 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277be1f-d0e7-478d-bdca-107dc8b636d2 None admin] RBAC: Adding query filter params (group.id=1ff36c94c8f6421da190b330cdbd9185, role.id=fec647282f5a42ab8490975708efad14, scope.project.id=9ec214e8e5724ce6b35fe31337cb0380) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:55.842017 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277be1f-d0e7-478d-bdca-107dc8b636d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:55.843903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277be1f-d0e7-478d-bdca-107dc8b636d2 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:55.850682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 209/425] 10.222.0.26 () {56 vars in 1376 bytes} [Mon May 9 09:33:55 2022] GET /identity/v3/role_assignments?group.id=1ff36c94c8f6421da190b330cdbd9185&scope.project.id=9ec214e8e5724ce6b35fe31337cb0380&role.id=fec647282f5a42ab8490975708efad14 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:55.858998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2d1b5-35ba-4655-bd8c-404829504294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:55.859944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2d1b5-35ba-4655-bd8c-404829504294 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:55.860309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2d1b5-35ba-4655-bd8c-404829504294 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:55.860684 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2d1b5-35ba-4655-bd8c-404829504294 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:55.884407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2d1b5-35ba-4655-bd8c-404829504294 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:55.892335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-18a2d1b5-35ba-4655-bd8c-404829504294 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:55.892335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:55.892335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:55.892335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:55.892335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:55.892335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:55.892335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:55.892335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:55.892335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:55.892335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:55.892335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:55.892335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:55.892335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:55.892335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:55.892335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:55.892335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:55.893275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:55.893275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:55.893275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:55.893275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:55.893275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:55.893275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:55.893275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:55.893275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:55.893275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:55.893275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:55.893275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:55.893275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:55.893275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:55.893275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:55.893275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:55.893275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:55.894075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:55.894075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:55.894075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:55.894075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:55.894075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:55.894075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:55.894075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:55.894075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:55.894075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:55.894075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:55.894075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 09:33:55.894075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:55.900872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 217/426] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:33:55.906520 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277be1f-d0e7-478d-bdca-107dc8b636d2 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:55.974082 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4960d-3bb8-4909-b787-fea1846f4df2 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:55.974655 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4960d-3bb8-4909-b787-fea1846f4df2 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:55.974655 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4960d-3bb8-4909-b787-fea1846f4df2 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:55.976479 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 218/427] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:55.994652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317d546-c1bb-43fb-baef-fe21d08a8901 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:55.994965 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317d546-c1bb-43fb-baef-fe21d08a8901 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:55.995198 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317d546-c1bb-43fb-baef-fe21d08a8901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:56.017208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-1317d546-c1bb-43fb-baef-fe21d08a8901 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:56.018432 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-1317d546-c1bb-43fb-baef-fe21d08a8901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:56.034031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277be1f-d0e7-478d-bdca-107dc8b636d2 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:56.072085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceae8d4-f2ec-41eb-86cd-ca6311bb30b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:56.073229 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceae8d4-f2ec-41eb-86cd-ca6311bb30b4 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:56.074151 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceae8d4-f2ec-41eb-86cd-ca6311bb30b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:56.074753 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceae8d4-f2ec-41eb-86cd-ca6311bb30b4 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:56.075722 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceae8d4-f2ec-41eb-86cd-ca6311bb30b4 None admin] RBAC: Adding query filter params (name=member) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:56.076225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceae8d4-f2ec-41eb-86cd-ca6311bb30b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:56.077882 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceae8d4-f2ec-41eb-86cd-ca6311bb30b4 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:56.088150 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 210/428] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:33:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:56.094260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceae8d4-f2ec-41eb-86cd-ca6311bb30b4 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:56.144878 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 219/429] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:33:56.165783 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1317d546-c1bb-43fb-baef-fe21d08a8901 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:56.198840 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceae8d4-f2ec-41eb-86cd-ca6311bb30b4 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:56.234713 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491ac54-0b16-4a03-bc3f-19e7c7d1020a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:56.235603 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491ac54-0b16-4a03-bc3f-19e7c7d1020a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:56.236021 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491ac54-0b16-4a03-bc3f-19e7c7d1020a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:56.236430 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491ac54-0b16-4a03-bc3f-19e7c7d1020a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:56.267651 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1317d546-c1bb-43fb-baef-fe21d08a8901 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:56.307571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a4b48-c490-47d7-bec9-f0240d9b7e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:56.308574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a4b48-c490-47d7-bec9-f0240d9b7e74 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:56.308963 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a4b48-c490-47d7-bec9-f0240d9b7e74 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:56.309413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a4b48-c490-47d7-bec9-f0240d9b7e74 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:56.310216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491ac54-0b16-4a03-bc3f-19e7c7d1020a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:56.311826 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491ac54-0b16-4a03-bc3f-19e7c7d1020a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:56.320243 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a4b48-c490-47d7-bec9-f0240d9b7e74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:56.328603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-833a4b48-c490-47d7-bec9-f0240d9b7e74 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:56.328603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:56.328603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:56.328603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:56.328603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:56.328603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:56.328603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:56.328603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:56.328603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:56.328603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:56.328603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:56.328603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:56.328603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:56.328603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:56.328603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:56.328603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:56.329590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:56.329590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:56.329590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:56.329590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:56.329590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:56.329590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:56.329590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:56.329590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:56.329590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:56.329590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:56.329590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:56.329590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:33:56.329590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:56.329590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:56.329590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:56.329590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:56.330400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:56.330400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:56.330400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:56.330400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:56.330400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:56.330400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:56.330400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:56.330400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:56.330400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:56.330400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:56.330400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:56.330400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:33:56.334694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 220/430] 10.222.0.26 () {56 vars in 1114 bytes} [Mon May 9 09:33:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:33:56.335836 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a4b48-c490-47d7-bec9-f0240d9b7e74 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:56.419066 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 211/431] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 09:33:56.424821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491ac54-0b16-4a03-bc3f-19e7c7d1020a None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:56.434058 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a4b48-c490-47d7-bec9-f0240d9b7e74 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:56.466528 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9165e7-3441-452f-b868-a8cb91db9fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:56.467374 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9165e7-3441-452f-b868-a8cb91db9fe4 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:56.467484 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9165e7-3441-452f-b868-a8cb91db9fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:56.467716 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9165e7-3441-452f-b868-a8cb91db9fe4 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:56.468463 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9165e7-3441-452f-b868-a8cb91db9fe4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:56.468793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9165e7-3441-452f-b868-a8cb91db9fe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:56.470153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9165e7-3441-452f-b868-a8cb91db9fe4 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:56.482714 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 221/432] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:33:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:56.484123 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9165e7-3441-452f-b868-a8cb91db9fe4 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:56.527056 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491ac54-0b16-4a03-bc3f-19e7c7d1020a None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:56.573005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7f1bb-dc39-4e01-bba4-cb9010b5b5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:56.574204 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7f1bb-dc39-4e01-bba4-cb9010b5b5f3 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:56.574694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7f1bb-dc39-4e01-bba4-cb9010b5b5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:56.575157 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7f1bb-dc39-4e01-bba4-cb9010b5b5f3 None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:56.576610 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7f1bb-dc39-4e01-bba4-cb9010b5b5f3 None admin] RBAC: Authorizing `identity:get_project(project_id=9c3d1e7bc10449e2957923e1cbb5e09e)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:56.578326 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7f1bb-dc39-4e01-bba4-cb9010b5b5f3 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:56.580520 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 212/433] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:56 2022] GET /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:56.580629 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9165e7-3441-452f-b868-a8cb91db9fe4 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:56.586365 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7f1bb-dc39-4e01-bba4-cb9010b5b5f3 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:56.615302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36291a69-9dc1-4d63-84e3-20502d9a4261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:56.616563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36291a69-9dc1-4d63-84e3-20502d9a4261 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:56.616563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36291a69-9dc1-4d63-84e3-20502d9a4261 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:56.616563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36291a69-9dc1-4d63-84e3-20502d9a4261 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:56.686801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36291a69-9dc1-4d63-84e3-20502d9a4261 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:56.688176 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36291a69-9dc1-4d63-84e3-20502d9a4261 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:56.697951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7f1bb-dc39-4e01-bba4-cb9010b5b5f3 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:56.732483 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41f2f8-1f3b-4333-9b61-a02101eefe85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:56.733731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41f2f8-1f3b-4333-9b61-a02101eefe85 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:56.734242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41f2f8-1f3b-4333-9b61-a02101eefe85 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:56.734702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41f2f8-1f3b-4333-9b61-a02101eefe85 None admin] PATH_INFO: `/v3/groups/1ff36c94c8f6421da190b330cdbd9185` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:56.736073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41f2f8-1f3b-4333-9b61-a02101eefe85 None admin] RBAC: Authorizing `identity:get_group(group_id=1ff36c94c8f6421da190b330cdbd9185)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:56.737745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41f2f8-1f3b-4333-9b61-a02101eefe85 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:56.739875 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 213/434] 10.222.0.26 () {56 vars in 1180 bytes} [Mon May 9 09:33:56 2022] GET /identity/v3/groups/1ff36c94c8f6421da190b330cdbd9185 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:56.746514 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41f2f8-1f3b-4333-9b61-a02101eefe85 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:56.794973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 222/435] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:56.796399 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36291a69-9dc1-4d63-84e3-20502d9a4261 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:56.852391 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41f2f8-1f3b-4333-9b61-a02101eefe85 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:56.886715 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9c7d2-098a-4e05-a944-22ab4a0a176b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:56.887815 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9c7d2-098a-4e05-a944-22ab4a0a176b None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:56.888291 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9c7d2-098a-4e05-a944-22ab4a0a176b None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:56.888791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9c7d2-098a-4e05-a944-22ab4a0a176b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:56.889870 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9c7d2-098a-4e05-a944-22ab4a0a176b None admin] RBAC: Adding query filter params (group.id=1ff36c94c8f6421da190b330cdbd9185, role.id=fec647282f5a42ab8490975708efad14, scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:56.890389 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9c7d2-098a-4e05-a944-22ab4a0a176b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:56.892034 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9c7d2-098a-4e05-a944-22ab4a0a176b None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:56.895407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36291a69-9dc1-4d63-84e3-20502d9a4261 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:56.898776 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 214/436] 10.222.0.26 () {56 vars in 1376 bytes} [Mon May 9 09:33:56 2022] GET /identity/v3/role_assignments?group.id=1ff36c94c8f6421da190b330cdbd9185&scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e&role.id=fec647282f5a42ab8490975708efad14 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:56.931107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaba2d6-ac64-4014-be44-0f46d5182d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:56.932053 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaba2d6-ac64-4014-be44-0f46d5182d26 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:56.932387 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaba2d6-ac64-4014-be44-0f46d5182d26 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:56.932655 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaba2d6-ac64-4014-be44-0f46d5182d26 None admin] PATH_INFO: `/v3/projects/9ec214e8e5724ce6b35fe31337cb0380` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:56.933943 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaba2d6-ac64-4014-be44-0f46d5182d26 None admin] RBAC: Authorizing `identity:get_project(project_id=9ec214e8e5724ce6b35fe31337cb0380)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:56.935417 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaba2d6-ac64-4014-be44-0f46d5182d26 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:56.941933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 223/437] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:56 2022] GET /identity/v3/projects/9ec214e8e5724ce6b35fe31337cb0380 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:56.943349 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9c7d2-098a-4e05-a944-22ab4a0a176b None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:56.997562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98418a-5544-4682-aa68-9b0149bd5b9a None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:56.998497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98418a-5544-4682-aa68-9b0149bd5b9a None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:56.998955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98418a-5544-4682-aa68-9b0149bd5b9a None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:57.000519 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 224/438] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:33:57.006430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbbdaa-7982-4084-836e-7a7f1eba256f None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:57.006971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbbdaa-7982-4084-836e-7a7f1eba256f None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:57.007407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbbdaa-7982-4084-836e-7a7f1eba256f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:57.033400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-28bbbdaa-7982-4084-836e-7a7f1eba256f None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:57.034953 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-28bbbdaa-7982-4084-836e-7a7f1eba256f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:57.069002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9c7d2-098a-4e05-a944-22ab4a0a176b None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:57.113927 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 225/439] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:33:57.118049 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a1906-3aed-4753-9fbc-594112f56252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:57.118916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a1906-3aed-4753-9fbc-594112f56252 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:57.119190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a1906-3aed-4753-9fbc-594112f56252 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:57.119454 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a1906-3aed-4753-9fbc-594112f56252 None admin] PATH_INFO: `/v3/groups/1ff36c94c8f6421da190b330cdbd9185` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:57.121127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7a1906-3aed-4753-9fbc-594112f56252 None admin] RBAC: Authorizing `identity:get_group(group_id=1ff36c94c8f6421da190b330cdbd9185)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:57.122585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7a1906-3aed-4753-9fbc-594112f56252 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:57.129158 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 215/440] 10.222.0.26 () {56 vars in 1180 bytes} [Mon May 9 09:33:56 2022] GET /identity/v3/groups/1ff36c94c8f6421da190b330cdbd9185 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:57.130547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbbdaa-7982-4084-836e-7a7f1eba256f None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:57.138265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a1906-3aed-4753-9fbc-594112f56252 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:57.245358 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbbdaa-7982-4084-836e-7a7f1eba256f None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:57.248825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a1906-3aed-4753-9fbc-594112f56252 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:57.277306 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7d182-17ed-4ede-8571-e56048fd26e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:57.277950 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7d182-17ed-4ede-8571-e56048fd26e6 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:57.278221 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7d182-17ed-4ede-8571-e56048fd26e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:57.278483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7d182-17ed-4ede-8571-e56048fd26e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:57.279303 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7d182-17ed-4ede-8571-e56048fd26e6 None admin] RBAC: Adding query filter params (group.id=1ff36c94c8f6421da190b330cdbd9185, role.id=0e3ae1b3fdce41c4a42a415efad3b84d, scope.project.id=9ec214e8e5724ce6b35fe31337cb0380) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:57.279605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7d182-17ed-4ede-8571-e56048fd26e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:57.281056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7d182-17ed-4ede-8571-e56048fd26e6 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:57.281638 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71723a2-d04b-40e9-a2ed-33356b508141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:57.282439 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71723a2-d04b-40e9-a2ed-33356b508141 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:57.282711 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71723a2-d04b-40e9-a2ed-33356b508141 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:57.282976 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71723a2-d04b-40e9-a2ed-33356b508141 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:57.286869 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 226/441] 10.222.0.26 () {56 vars in 1376 bytes} [Mon May 9 09:33:57 2022] GET /identity/v3/role_assignments?group.id=1ff36c94c8f6421da190b330cdbd9185&scope.project.id=9ec214e8e5724ce6b35fe31337cb0380&role.id=0e3ae1b3fdce41c4a42a415efad3b84d => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:57.293319 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71723a2-d04b-40e9-a2ed-33356b508141 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:57.301734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-a71723a2-d04b-40e9-a2ed-33356b508141 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:57.301734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:57.301734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:57.301734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:57.301734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:57.301734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:57.301734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:57.301734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:57.301734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:57.301734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:57.301734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:57.301734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:57.301734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:57.301734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:57.301734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:57.301734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:57.302704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:57.302704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:57.302704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:57.302704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:57.302704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:57.302704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:57.302704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:57.302704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:57.302704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:57.302704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:57.302704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:57.302704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:33:57.302704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:57.302704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:57.302704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:57.302704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:57.303550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:57.303550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:57.303550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:57.303550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:57.303550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:57.303550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:57.303550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:57.303550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:57.303550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:57.303550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:57.303550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:33:57.303550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:33:57.304844 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 216/442] 10.222.0.26 () {56 vars in 1114 bytes} [Mon May 9 09:33:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:33:57.309125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7d182-17ed-4ede-8571-e56048fd26e6 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:57.413635 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7d182-17ed-4ede-8571-e56048fd26e6 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:57.450201 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad34749-e5ea-4f72-a07f-ee9d9103dea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:57.450888 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad34749-e5ea-4f72-a07f-ee9d9103dea5 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:57.451168 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad34749-e5ea-4f72-a07f-ee9d9103dea5 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:57.451440 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad34749-e5ea-4f72-a07f-ee9d9103dea5 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:57.452192 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad34749-e5ea-4f72-a07f-ee9d9103dea5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:57.452493 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad34749-e5ea-4f72-a07f-ee9d9103dea5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:57.453925 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad34749-e5ea-4f72-a07f-ee9d9103dea5 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:57.467333 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 227/443] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:33:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:57.468547 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71723a2-d04b-40e9-a2ed-33356b508141 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:57.571843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71723a2-d04b-40e9-a2ed-33356b508141 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:57.606892 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303be18-9fd2-49c7-b946-aa2fcb8c665a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:57.607572 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303be18-9fd2-49c7-b946-aa2fcb8c665a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:57.607847 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303be18-9fd2-49c7-b946-aa2fcb8c665a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:57.608114 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303be18-9fd2-49c7-b946-aa2fcb8c665a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:57.680083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303be18-9fd2-49c7-b946-aa2fcb8c665a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:57.681928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303be18-9fd2-49c7-b946-aa2fcb8c665a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:57.783012 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 217/444] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:57.788604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad34749-e5ea-4f72-a07f-ee9d9103dea5 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:57.790011 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97943e7c-de69-44b3-865b-61e768f0c4d3 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:57.790301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97943e7c-de69-44b3-865b-61e768f0c4d3 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:57.790577 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97943e7c-de69-44b3-865b-61e768f0c4d3 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:57.792092 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 218/445] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:33:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:33:57.797282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90d733-3091-4007-87d2-705323929ee9 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:57.797566 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90d733-3091-4007-87d2-705323929ee9 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:57.797841 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90d733-3091-4007-87d2-705323929ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:57.815707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-2f90d733-3091-4007-87d2-705323929ee9 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:33:57.816875 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-2f90d733-3091-4007-87d2-705323929ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:33:57.881387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 219/446] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:33:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:33:57.896146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad34749-e5ea-4f72-a07f-ee9d9103dea5 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:57.904247 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90d733-3091-4007-87d2-705323929ee9 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:57.927335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fed4b-5af9-456b-ab13-2e940272eec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:57.928192 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fed4b-5af9-456b-ab13-2e940272eec2 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:57.928459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fed4b-5af9-456b-ab13-2e940272eec2 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:57.928752 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fed4b-5af9-456b-ab13-2e940272eec2 None admin] PATH_INFO: `/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:57.929919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91fed4b-5af9-456b-ab13-2e940272eec2 None admin] RBAC: Authorizing `identity:get_project(project_id=9c3d1e7bc10449e2957923e1cbb5e09e)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:57.931374 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91fed4b-5af9-456b-ab13-2e940272eec2 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:57.933145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 228/447] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:57 2022] GET /identity/v3/projects/9c3d1e7bc10449e2957923e1cbb5e09e => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:57.938419 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fed4b-5af9-456b-ab13-2e940272eec2 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:57.995362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90d733-3091-4007-87d2-705323929ee9 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:58.027208 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2e234-39a0-442c-8898-93d6143cb559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:58.028001 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2e234-39a0-442c-8898-93d6143cb559 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:58.028278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2e234-39a0-442c-8898-93d6143cb559 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:58.028546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2e234-39a0-442c-8898-93d6143cb559 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:58.032232 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fed4b-5af9-456b-ab13-2e940272eec2 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:58.038400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2e234-39a0-442c-8898-93d6143cb559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:33:58.046063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-c6c2e234-39a0-442c-8898-93d6143cb559 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:33:58.046063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:33:58.046063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:33:58.046063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:33:58.046063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:33:58.046063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:33:58.046063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:33:58.046063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:33:58.046063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:33:58.046063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:33:58.046063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:33:58.046063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:33:58.046063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:33:58.046063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:33:58.046063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:33:58.046063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:33:58.046966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:33:58.046966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:33:58.046966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:33:58.046966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:33:58.046966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:33:58.046966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:33:58.046966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:33:58.046966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:33:58.046966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:33:58.046966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:33:58.046966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:33:58.046966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:33:58.046966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:33:58.046966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:33:58.046966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:33:58.046966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:33:58.047801 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:33:58.047801 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:33:58.047801 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:33:58.047801 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:33:58.047801 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:33:58.047801 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:33:58.047801 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:33:58.047801 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:33:58.047801 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:33:58.047801 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:33:58.047801 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:33:58.047801 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:33:58.047801 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 220/448] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:33:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:33:58.052952 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2e234-39a0-442c-8898-93d6143cb559 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:58.064406 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f022597b-21aa-4dfd-ba3d-b1fa35a27433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:58.065277 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022597b-21aa-4dfd-ba3d-b1fa35a27433 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:58.065548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022597b-21aa-4dfd-ba3d-b1fa35a27433 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:58.065861 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022597b-21aa-4dfd-ba3d-b1fa35a27433 None admin] PATH_INFO: `/v3/groups/1ff36c94c8f6421da190b330cdbd9185` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:58.066957 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f022597b-21aa-4dfd-ba3d-b1fa35a27433 None admin] RBAC: Authorizing `identity:get_group(group_id=1ff36c94c8f6421da190b330cdbd9185)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:58.068377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f022597b-21aa-4dfd-ba3d-b1fa35a27433 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:58.069989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 229/449] 10.222.0.26 () {56 vars in 1180 bytes} [Mon May 9 09:33:57 2022] GET /identity/v3/groups/1ff36c94c8f6421da190b330cdbd9185 => generated 226 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:58.075726 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f022597b-21aa-4dfd-ba3d-b1fa35a27433 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:58.146322 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2e234-39a0-442c-8898-93d6143cb559 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:58.165762 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f022597b-21aa-4dfd-ba3d-b1fa35a27433 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:58.177803 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbca4c9-cf27-4278-bdf1-19ce05b395d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:58.178472 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbca4c9-cf27-4278-bdf1-19ce05b395d2 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:58.178749 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbca4c9-cf27-4278-bdf1-19ce05b395d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:58.179014 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbca4c9-cf27-4278-bdf1-19ce05b395d2 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:58.179753 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbca4c9-cf27-4278-bdf1-19ce05b395d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:58.180052 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbca4c9-cf27-4278-bdf1-19ce05b395d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:58.181449 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbca4c9-cf27-4278-bdf1-19ce05b395d2 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:58.193699 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 221/450] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:33:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:58.194650 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbca4c9-cf27-4278-bdf1-19ce05b395d2 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:58.197738 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b842c-8269-45a0-82a4-38f6d95c21c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:58.198368 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b842c-8269-45a0-82a4-38f6d95c21c2 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:58.198642 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b842c-8269-45a0-82a4-38f6d95c21c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:58.198913 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b842c-8269-45a0-82a4-38f6d95c21c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:58.199746 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b842c-8269-45a0-82a4-38f6d95c21c2 None admin] RBAC: Adding query filter params (group.id=1ff36c94c8f6421da190b330cdbd9185, role.id=0e3ae1b3fdce41c4a42a415efad3b84d, scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:58.200053 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b842c-8269-45a0-82a4-38f6d95c21c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:58.201497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b842c-8269-45a0-82a4-38f6d95c21c2 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:58.207255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 230/451] 10.222.0.26 () {56 vars in 1376 bytes} [Mon May 9 09:33:58 2022] GET /identity/v3/role_assignments?group.id=1ff36c94c8f6421da190b330cdbd9185&scope.project.id=9c3d1e7bc10449e2957923e1cbb5e09e&role.id=0e3ae1b3fdce41c4a42a415efad3b84d => generated 611 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:33:58.298283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbca4c9-cf27-4278-bdf1-19ce05b395d2 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:58.334116 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f86770-f768-4d42-8b17-3477298ac5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:58.334674 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f86770-f768-4d42-8b17-3477298ac5a3 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:58.334950 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f86770-f768-4d42-8b17-3477298ac5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:58.335220 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f86770-f768-4d42-8b17-3477298ac5a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:58.405784 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f86770-f768-4d42-8b17-3477298ac5a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:58.407240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f86770-f768-4d42-8b17-3477298ac5a3 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:58.501859 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 222/452] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:33:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:33:58.507609 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b842c-8269-45a0-82a4-38f6d95c21c2 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:58.608336 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b842c-8269-45a0-82a4-38f6d95c21c2 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:58.643125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc5092-c70d-4063-870b-c79773dce729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:58.643980 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc5092-c70d-4063-870b-c79773dce729 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:58.644258 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc5092-c70d-4063-870b-c79773dce729 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:58.644529 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc5092-c70d-4063-870b-c79773dce729 None admin] PATH_INFO: `/v3/projects/03d8c548fb514d70b477284ceb7b3252` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:58.645797 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc5092-c70d-4063-870b-c79773dce729 None admin] RBAC: Authorizing `identity:get_project(project_id=03d8c548fb514d70b477284ceb7b3252)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:58.647237 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc5092-c70d-4063-870b-c79773dce729 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:58.648980 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 231/453] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:33:58 2022] GET /identity/v3/projects/03d8c548fb514d70b477284ceb7b3252 => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:58.654826 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f86770-f768-4d42-8b17-3477298ac5a3 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:58.752056 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f86770-f768-4d42-8b17-3477298ac5a3 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:58.786243 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9af4d-d210-4d6b-8991-be850f86a517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:58.787080 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9af4d-d210-4d6b-8991-be850f86a517 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:58.787350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9af4d-d210-4d6b-8991-be850f86a517 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:58.787613 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9af4d-d210-4d6b-8991-be850f86a517 None admin] PATH_INFO: `/v3/groups/e9a1c2c9bfb1473bb2947e144ceafa64` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:58.788933 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da9af4d-d210-4d6b-8991-be850f86a517 None admin] RBAC: Authorizing `identity:get_group(group_id=e9a1c2c9bfb1473bb2947e144ceafa64)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:58.790381 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da9af4d-d210-4d6b-8991-be850f86a517 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:58.792075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 223/454] 10.222.0.26 () {56 vars in 1180 bytes} [Mon May 9 09:33:58 2022] GET /identity/v3/groups/e9a1c2c9bfb1473bb2947e144ceafa64 => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:33:58.798292 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc5092-c70d-4063-870b-c79773dce729 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:33:58.899240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc5092-c70d-4063-870b-c79773dce729 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:33:58.940339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93c087-bd9a-4a1f-a1cf-0c8abba292a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:33:58.941000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93c087-bd9a-4a1f-a1cf-0c8abba292a6 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:33:58.941278 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93c087-bd9a-4a1f-a1cf-0c8abba292a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:33:58.941547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93c087-bd9a-4a1f-a1cf-0c8abba292a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:33:58.942352 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b93c087-bd9a-4a1f-a1cf-0c8abba292a6 None admin] RBAC: Adding query filter params (group.id=e9a1c2c9bfb1473bb2947e144ceafa64, role.id=70ac549382b14b2cbb8b35e5dfc648d4, scope.project.id=03d8c548fb514d70b477284ceb7b3252) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:33:58.942658 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b93c087-bd9a-4a1f-a1cf-0c8abba292a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:33:58.944085 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b93c087-bd9a-4a1f-a1cf-0c8abba292a6 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:33:58.952013 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 232/455] 10.222.0.26 () {56 vars in 1376 bytes} [Mon May 9 09:33:58 2022] GET /identity/v3/role_assignments?group.id=e9a1c2c9bfb1473bb2947e144ceafa64&scope.project.id=03d8c548fb514d70b477284ceb7b3252&role.id=70ac549382b14b2cbb8b35e5dfc648d4 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:01.579760 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d781a-105b-4827-a5a4-f79fb3564d78 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:01.580767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d781a-105b-4827-a5a4-f79fb3564d78 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:01.581309 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d781a-105b-4827-a5a4-f79fb3564d78 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:01.583158 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 224/456] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:01.588527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342a51f-d0ec-4026-a63f-a58935b95abd None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:01.589469 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342a51f-d0ec-4026-a63f-a58935b95abd None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:01.589937 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342a51f-d0ec-4026-a63f-a58935b95abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:01.613800 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-2342a51f-d0ec-4026-a63f-a58935b95abd None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:01.615399 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-2342a51f-d0ec-4026-a63f-a58935b95abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:01.709570 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 233/457] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:01.742577 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d781a-105b-4827-a5a4-f79fb3564d78 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:01.899554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d781a-105b-4827-a5a4-f79fb3564d78 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:01.949386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5a1c0-0463-4292-ab66-7458ccd1f0c5 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:01.950043 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5a1c0-0463-4292-ab66-7458ccd1f0c5 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:01.950473 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef013849-90bd-452d-b9c1-6ab92383d24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:01.950752 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5a1c0-0463-4292-ab66-7458ccd1f0c5 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:01.951267 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef013849-90bd-452d-b9c1-6ab92383d24e None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:01.951539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef013849-90bd-452d-b9c1-6ab92383d24e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:01.951807 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef013849-90bd-452d-b9c1-6ab92383d24e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:01.952974 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 234/458] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:34:01.958609 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006cc221-0c8e-4238-891c-85b05b445bdf None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:01.959073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006cc221-0c8e-4238-891c-85b05b445bdf None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:01.959483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006cc221-0c8e-4238-891c-85b05b445bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:01.967592 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef013849-90bd-452d-b9c1-6ab92383d24e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:01.970607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef013849-90bd-452d-b9c1-6ab92383d24e None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:01.996887 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-006cc221-0c8e-4238-891c-85b05b445bdf None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:01.998114 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-006cc221-0c8e-4238-891c-85b05b445bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:02.018306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-ef013849-90bd-452d-b9c1-6ab92383d24e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:02.018306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:02.018306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:02.018306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:02.018306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:02.018306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:02.018306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:02.018306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:02.018306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:02.018306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:02.018306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:02.018306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:02.018306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:02.018306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:02.018306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:02.019487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:02.019487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:02.019487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:02.019487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:02.019487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:02.019487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:02.019487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:02.019487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:02.019487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:02.019487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:02.019487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:02.019487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:02.019487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:02.019487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:02.019487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:02.019487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:02.019487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:02.020766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:02.020766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:02.020766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:02.020766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:02.020766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 225/459] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:02.040233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef013849-90bd-452d-b9c1-6ab92383d24e None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:02.120301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 235/460] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:34:02.164783 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006cc221-0c8e-4238-891c-85b05b445bdf None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:02.197509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef013849-90bd-452d-b9c1-6ab92383d24e None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:02.234524 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015274c-15c8-49c8-9018-a9414010effd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:02.235180 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015274c-15c8-49c8-9018-a9414010effd None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:02.235459 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015274c-15c8-49c8-9018-a9414010effd None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:02.235731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015274c-15c8-49c8-9018-a9414010effd None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:02.236453 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015274c-15c8-49c8-9018-a9414010effd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:02.236801 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015274c-15c8-49c8-9018-a9414010effd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:02.238172 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015274c-15c8-49c8-9018-a9414010effd None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:02.253074 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 226/461] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:02.253074 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13b14c-481e-4317-919b-26ea118d575d None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:02.253074 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13b14c-481e-4317-919b-26ea118d575d None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:02.253074 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13b14c-481e-4317-919b-26ea118d575d None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:02.257734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 227/462] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:02.257734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a4719-e2b1-4797-b5c0-4af157b3350b None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:02.257734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a4719-e2b1-4797-b5c0-4af157b3350b None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:02.257734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a4719-e2b1-4797-b5c0-4af157b3350b None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:02.257734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 228/463] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:02.257734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a4719-e2b1-4797-b5c0-4af157b3350b None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:02.283436 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006cc221-0c8e-4238-891c-85b05b445bdf None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:02.315366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eea4c5-05d6-4825-8b61-f402d3e8142f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:02.316155 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eea4c5-05d6-4825-8b61-f402d3e8142f None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:02.316430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eea4c5-05d6-4825-8b61-f402d3e8142f None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:02.316714 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eea4c5-05d6-4825-8b61-f402d3e8142f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:02.326784 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eea4c5-05d6-4825-8b61-f402d3e8142f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:02.328223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eea4c5-05d6-4825-8b61-f402d3e8142f None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:02.344972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-01eea4c5-05d6-4825-8b61-f402d3e8142f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:02.344972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:02.344972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:02.344972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:02.344972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:02.344972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:02.344972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:02.344972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:02.344972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:02.344972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:02.344972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:02.344972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:02.344972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:02.344972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:02.344972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:02.346048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:02.346048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:02.346048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:02.346048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:02.346048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:02.346048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:02.346048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:02.346048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:02.346048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:02.346048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:02.346048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:02.346048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:02.346048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:02.346048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:02.346048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:02.346048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:02.346048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:02.347165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:02.347165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:02.347165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:02.347165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:02.347165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 236/464] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:02.347475 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa9e06-b478-4418-a577-3e20ea76f55d None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:02.347801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa9e06-b478-4418-a577-3e20ea76f55d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:02.347905 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa9e06-b478-4418-a577-3e20ea76f55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:02.358158 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a4719-e2b1-4797-b5c0-4af157b3350b None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:02.363518 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-d7fa9e06-b478-4418-a577-3e20ea76f55d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:02.364713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-d7fa9e06-b478-4418-a577-3e20ea76f55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:02.392265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53362262-6b5e-4c86-832b-d97ccabebb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:02.392972 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53362262-6b5e-4c86-832b-d97ccabebb9c None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:02.393232 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53362262-6b5e-4c86-832b-d97ccabebb9c None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:02.393505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53362262-6b5e-4c86-832b-d97ccabebb9c None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:02.394208 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53362262-6b5e-4c86-832b-d97ccabebb9c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:02.395642 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53362262-6b5e-4c86-832b-d97ccabebb9c None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:02.424791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 229/465] 10.222.0.26 () {60 vars in 1132 bytes} [Mon May 9 09:34:02 2022] POST /identity/v3/users => generated 246 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:34:02.424791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf68deb-f2e7-4937-ad1c-338d3743672a None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:02.424791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf68deb-f2e7-4937-ad1c-338d3743672a None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:02.424791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf68deb-f2e7-4937-ad1c-338d3743672a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:02.431082 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 237/466] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:02.431989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa9e06-b478-4418-a577-3e20ea76f55d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:02.465869 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-7bf68deb-f2e7-4937-ad1c-338d3743672a None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:02.467105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-7bf68deb-f2e7-4937-ad1c-338d3743672a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:02.534184 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa9e06-b478-4418-a577-3e20ea76f55d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:02.538542 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 230/467] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:02.539436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf68deb-f2e7-4937-ad1c-338d3743672a None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:02.579644 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3d32e-d6b5-4f4f-8080-089ebe966b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:02.580618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3d32e-d6b5-4f4f-8080-089ebe966b61 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:02.580937 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3d32e-d6b5-4f4f-8080-089ebe966b61 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:02.581201 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3d32e-d6b5-4f4f-8080-089ebe966b61 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:02.581927 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc3d32e-d6b5-4f4f-8080-089ebe966b61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:02.582223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc3d32e-d6b5-4f4f-8080-089ebe966b61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:02.583602 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc3d32e-d6b5-4f4f-8080-089ebe966b61 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:02.596366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 238/468] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:02.597252 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3d32e-d6b5-4f4f-8080-089ebe966b61 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:02.642767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf68deb-f2e7-4937-ad1c-338d3743672a None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:02.678891 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051da01-e220-46fb-af96-4aa2f5956ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:02.679988 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051da01-e220-46fb-af96-4aa2f5956ba3 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:02.680451 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051da01-e220-46fb-af96-4aa2f5956ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:02.680761 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051da01-e220-46fb-af96-4aa2f5956ba3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:02.691214 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051da01-e220-46fb-af96-4aa2f5956ba3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:02.692613 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051da01-e220-46fb-af96-4aa2f5956ba3 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:02.701947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3d32e-d6b5-4f4f-8080-089ebe966b61 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:02.711265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-9051da01-e220-46fb-af96-4aa2f5956ba3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:02.711265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:02.711265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:02.711265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:02.711265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:02.711265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:02.711265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:02.711265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:02.711265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:02.711265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:02.711265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:02.711265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:02.711265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:02.711265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:02.711265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:02.712128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:02.712128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:02.712128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:02.712128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:02.712128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:02.712128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:02.712128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:02.712128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:02.712128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:02.712128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:02.712128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:02.712128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:02.712128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:02.712128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:02.712128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:02.712128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:02.712128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:02.713378 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:02.713378 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:02.713378 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:02.713378 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:02.718146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 231/469] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:02.718985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051da01-e220-46fb-af96-4aa2f5956ba3 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:02.736859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1558c-d063-4a72-b12b-22fce1ddeee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:02.737984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1558c-d063-4a72-b12b-22fce1ddeee3 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:02.738454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1558c-d063-4a72-b12b-22fce1ddeee3 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:02.738914 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1558c-d063-4a72-b12b-22fce1ddeee3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:02.749611 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d1558c-d063-4a72-b12b-22fce1ddeee3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:02.751360 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d1558c-d063-4a72-b12b-22fce1ddeee3 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:02.769959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-b3d1558c-d063-4a72-b12b-22fce1ddeee3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:02.769959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:02.769959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:02.769959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:02.769959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:02.769959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:02.769959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:02.769959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:02.769959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:02.769959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:02.769959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:02.769959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:02.769959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:02.769959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:02.769959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:02.771039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:02.771039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:02.771039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:02.771039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:02.771039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:02.771039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:02.771039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:02.771039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:02.771039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:02.771039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:02.771039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:02.771039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:02.771039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:02.771039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:02.771039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:02.771039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:02.771039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:02.772087 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:02.772087 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:02.772087 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:02.772087 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:02.773928 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 239/470] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:02.774852 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1558c-d063-4a72-b12b-22fce1ddeee3 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:02.819604 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051da01-e220-46fb-af96-4aa2f5956ba3 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:02.853652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2530b-5d67-4997-a13d-2ff001b78156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:02.854388 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2530b-5d67-4997-a13d-2ff001b78156 None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:02.854669 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2530b-5d67-4997-a13d-2ff001b78156 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:02.854943 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2530b-5d67-4997-a13d-2ff001b78156 None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:02.855832 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2530b-5d67-4997-a13d-2ff001b78156 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:02.857630 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2530b-5d67-4997-a13d-2ff001b78156 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:02.871463 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1558c-d063-4a72-b12b-22fce1ddeee3 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:02.889058 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 232/471] 10.222.0.26 () {60 vars in 1132 bytes} [Mon May 9 09:34:02 2022] POST /identity/v3/users => generated 248 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:34:02.890146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2530b-5d67-4997-a13d-2ff001b78156 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:02.910397 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa6bc2-5749-47be-920a-20170b760042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:02.911383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa6bc2-5749-47be-920a-20170b760042 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:02.911827 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa6bc2-5749-47be-920a-20170b760042 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:02.912274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa6bc2-5749-47be-920a-20170b760042 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:02.913214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa6bc2-5749-47be-920a-20170b760042 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:02.913664 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa6bc2-5749-47be-920a-20170b760042 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:02.915285 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa6bc2-5749-47be-920a-20170b760042 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:02.928172 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 240/472] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:02.934490 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa6bc2-5749-47be-920a-20170b760042 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:03.007685 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2530b-5d67-4997-a13d-2ff001b78156 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:03.054637 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f96af-95dd-49e2-8c00-60a38d0b3060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:03.055575 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f96af-95dd-49e2-8c00-60a38d0b3060 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:03.056081 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f96af-95dd-49e2-8c00-60a38d0b3060 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:03.056603 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f96af-95dd-49e2-8c00-60a38d0b3060 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:03.057591 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f96af-95dd-49e2-8c00-60a38d0b3060 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:03.058213 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f96af-95dd-49e2-8c00-60a38d0b3060 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:03.059842 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f96af-95dd-49e2-8c00-60a38d0b3060 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:03.075006 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 233/473] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:03.078846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa6bc2-5749-47be-920a-20170b760042 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:03.081478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f96af-95dd-49e2-8c00-60a38d0b3060 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:03.133079 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40edf8e-f685-495e-9197-b84de3119beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:03.134001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40edf8e-f685-495e-9197-b84de3119beb None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:03.134438 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40edf8e-f685-495e-9197-b84de3119beb None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:03.134852 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40edf8e-f685-495e-9197-b84de3119beb None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:03.135882 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40edf8e-f685-495e-9197-b84de3119beb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:03.137506 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40edf8e-f685-495e-9197-b84de3119beb None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:03.162239 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 241/474] 10.222.0.26 () {60 vars in 1132 bytes} [Mon May 9 09:34:02 2022] POST /identity/v3/users => generated 249 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:34:03.163860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3dcfce-9a4c-40ec-a387-4b851710e870 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:03.164364 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3dcfce-9a4c-40ec-a387-4b851710e870 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:03.164837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3dcfce-9a4c-40ec-a387-4b851710e870 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:03.166798 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 242/475] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:03.181959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876551d-df80-41ab-9eee-6207cb4d2f52 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:03.182244 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876551d-df80-41ab-9eee-6207cb4d2f52 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:03.182514 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876551d-df80-41ab-9eee-6207cb4d2f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:03.202486 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-0876551d-df80-41ab-9eee-6207cb4d2f52 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:03.203603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-0876551d-df80-41ab-9eee-6207cb4d2f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:03.260980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f96af-95dd-49e2-8c00-60a38d0b3060 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:03.277532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 243/476] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:03.308367 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ff968-ea2b-48a1-aaf4-109a392ed72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:03.309089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ff968-ea2b-48a1-aaf4-109a392ed72b None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:03.309364 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ff968-ea2b-48a1-aaf4-109a392ed72b None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:03.309629 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ff968-ea2b-48a1-aaf4-109a392ed72b None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:03.310384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1ff968-ea2b-48a1-aaf4-109a392ed72b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:03.311606 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876551d-df80-41ab-9eee-6207cb4d2f52 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:03.317504 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1ff968-ea2b-48a1-aaf4-109a392ed72b None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:03.350639 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 234/477] 10.222.0.26 () {60 vars in 1132 bytes} [Mon May 9 09:34:03 2022] POST /identity/v3/users => generated 248 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:34:03.448542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876551d-df80-41ab-9eee-6207cb4d2f52 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:03.488650 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875eda7-20a6-4d48-a749-7b1a5654e486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:03.489285 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875eda7-20a6-4d48-a749-7b1a5654e486 None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:03.489560 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875eda7-20a6-4d48-a749-7b1a5654e486 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:03.489829 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875eda7-20a6-4d48-a749-7b1a5654e486 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:03.490588 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0875eda7-20a6-4d48-a749-7b1a5654e486 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:03.491977 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0875eda7-20a6-4d48-a749-7b1a5654e486 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:03.501471 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.sql.core [None req-0875eda7-20a6-4d48-a749-7b1a5654e486 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 09:34:03.501471 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 09 09:34:03.501471 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [parameters: {'id': '5495a1d06fe541babf676cbbe4de0268', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 09 09:34:03.501471 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86598) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 09:34:03.518646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-0875eda7-20a6-4d48-a749-7b1a5654e486 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 09 09:34:03.518646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:03.518646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 09:34:03.518646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 09:34:03.518646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 09:34:03.518646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 09:34:03.518646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 09:34:03.518646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application result = self._query(query) May 09 09:34:03.518646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 09:34:03.518646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application conn.query(q) May 09 09:34:03.518646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 09:34:03.518646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 09:34:03.518646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 09:34:03.518646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application result.read() May 09 09:34:03.518646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 09:34:03.519636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 09:34:03.519636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 09:34:03.519636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 09:34:03.519636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 09:34:03.519636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 09:34:03.519636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 09:34:03.519636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 09:34:03.519636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 09:34:03.519636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:03.519636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 09:34:03.519636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:03.519636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:03.519636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 09:34:03.519636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 09:34:03.519636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 09 09:34:03.519636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return ref.to_dict() May 09 09:34:03.519636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 09:34:03.519636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application next(self.gen) May 09 09:34:03.520548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 09:34:03.520548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application yield resource May 09 09:34:03.520548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 09:34:03.520548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application next(self.gen) May 09 09:34:03.520548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 09:34:03.520548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self.session.rollback() May 09 09:34:03.520548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 09:34:03.520548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self.force_reraise() May 09 09:34:03.520548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 09:34:03.520548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise self.value May 09 09:34:03.520548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 09:34:03.520548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 09:34:03.520548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 09:34:03.520548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application session.commit() May 09 09:34:03.520548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 09:34:03.520548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 09:34:03.520548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 09:34:03.521449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._prepare_impl() May 09 09:34:03.521449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 09:34:03.521449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self.session.flush() May 09 09:34:03.521449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 09:34:03.521449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._flush(objects) May 09 09:34:03.521449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 09:34:03.521449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 09:34:03.521449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 09:34:03.521449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application compat.raise_( May 09 09:34:03.521449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 09:34:03.521449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception May 09 09:34:03.521449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 09:34:03.521449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application flush_context.execute() May 09 09:34:03.521449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 09:34:03.521449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rec.execute(self) May 09 09:34:03.521449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 09:34:03.521449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 09:34:03.522357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 09:34:03.522357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 09:34:03.522357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 09 09:34:03.522357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application c = connection._execute_20( May 09 09:34:03.522357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 09:34:03.522357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 09:34:03.522357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 09:34:03.522357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 09:34:03.522357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 09:34:03.522357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 09:34:03.522357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 09:34:03.522357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 09:34:03.522357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 09:34:03.522357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 09:34:03.522357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 09:34:03.522357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception May 09 09:34:03.523189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 09:34:03.523189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 09:34:03.523189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 09:34:03.523189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 09:34:03.523189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 09:34:03.523189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application result = self._query(query) May 09 09:34:03.523189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 09:34:03.523189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application conn.query(q) May 09 09:34:03.523189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 09:34:03.523189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 09:34:03.523189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 09:34:03.523189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application result.read() May 09 09:34:03.523189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 09:34:03.523189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 09:34:03.523189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 09:34:03.523189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 09:34:03.524003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 09:34:03.524003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 09:34:03.524003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 09:34:03.524003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 09:34:03.524003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 09:34:03.524003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 09 09:34:03.524003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [parameters: {'id': '5495a1d06fe541babf676cbbe4de0268', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 09 09:34:03.524003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 09:34:03.524003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:03.524003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:34:03.524003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:03.524003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:03.524003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:03.524003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:03.524003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:03.524003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:03.526326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:03.526326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:03.526326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:03.526326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:03.526326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:03.526326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:03.526326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 09 09:34:03.526326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 09 09:34:03.526326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:03.526326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:03.526326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 09 09:34:03.526326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 09 09:34:03.526326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 09:34:03.526326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 09:34:03.526326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 09 09:34:03.526326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:03.528371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 244/478] 10.222.0.26 () {60 vars in 1131 bytes} [Mon May 9 09:34:03 2022] POST /identity/v3/roles => generated 144 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:03.535269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ff968-ea2b-48a1-aaf4-109a392ed72b None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:03.648657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ff968-ea2b-48a1-aaf4-109a392ed72b None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:03.683201 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fe89b-3bb3-46a7-af80-c9d5045f3960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:03.684070 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fe89b-3bb3-46a7-af80-c9d5045f3960 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:03.684345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fe89b-3bb3-46a7-af80-c9d5045f3960 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:03.684612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fe89b-3bb3-46a7-af80-c9d5045f3960 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:03.695397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fe89b-3bb3-46a7-af80-c9d5045f3960 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:03.703633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-354fe89b-3bb3-46a7-af80-c9d5045f3960 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:34:03.703633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:03.703633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:03.703633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:03.703633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:03.703633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:03.703633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:03.703633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:03.703633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:03.703633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:03.703633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:03.703633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:03.703633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:03.703633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:03.703633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:03.703633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:03.705820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:03.705820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:03.705820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:03.705820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:03.705820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:03.705820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:03.705820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:03.705820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:03.705820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:03.705820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:03.705820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:03.705820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:03.705820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:03.705820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:03.705820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:03.705820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:03.707743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:03.707743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:03.707743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:03.707743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:03.707743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:03.707743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:03.707743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:03.707743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:03.707743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:03.707743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:03.707743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 09:34:03.707743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:03.707743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 235/479] 10.222.0.26 () {56 vars in 1114 bytes} [Mon May 9 09:34:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:03.717772 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875eda7-20a6-4d48-a749-7b1a5654e486 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:03.820092 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875eda7-20a6-4d48-a749-7b1a5654e486 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:03.859686 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2b457-9eee-4a02-b7ae-d19390114cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:03.860351 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2b457-9eee-4a02-b7ae-d19390114cb3 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:03.860624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2b457-9eee-4a02-b7ae-d19390114cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:03.860913 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2b457-9eee-4a02-b7ae-d19390114cb3 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:03.861659 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2b457-9eee-4a02-b7ae-d19390114cb3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:03.861960 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2b457-9eee-4a02-b7ae-d19390114cb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:03.863324 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2b457-9eee-4a02-b7ae-d19390114cb3 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:03.876235 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 245/480] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:05.101352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f5d9a-ac69-4491-83c7-f27dfc55b4f5 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:05.101352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f5d9a-ac69-4491-83c7-f27dfc55b4f5 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:05.101352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f5d9a-ac69-4491-83c7-f27dfc55b4f5 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:05.101352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 236/481] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:05.113883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f29b0-2f59-4c20-badb-7d65cc7ab20a None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:05.113883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f29b0-2f59-4c20-badb-7d65cc7ab20a None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:05.113883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f29b0-2f59-4c20-badb-7d65cc7ab20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:05.138615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-721f29b0-2f59-4c20-badb-7d65cc7ab20a None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:05.140151 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-721f29b0-2f59-4c20-badb-7d65cc7ab20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:05.224365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 246/482] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:34:05.259600 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f5d9a-ac69-4491-83c7-f27dfc55b4f5 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:05.399371 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f5d9a-ac69-4491-83c7-f27dfc55b4f5 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:05.442837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcea9a-61cc-451c-bc83-9cabba73d6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:05.443648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcea9a-61cc-451c-bc83-9cabba73d6f5 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:05.443932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcea9a-61cc-451c-bc83-9cabba73d6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:05.444207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcea9a-61cc-451c-bc83-9cabba73d6f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:05.457751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fcea9a-61cc-451c-bc83-9cabba73d6f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:05.465703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-35fcea9a-61cc-451c-bc83-9cabba73d6f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:05.465703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:05.465703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:05.465703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:05.465703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:05.465703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:05.465703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:05.465703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:05.465703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:05.465703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:05.465703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:05.465703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:05.465703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:05.465703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:05.465703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:05.465703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:05.466612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:05.466612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:05.466612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:05.466612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:05.466612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:05.466612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:05.466612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:05.466612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:05.466612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:05.466612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:05.466612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:05.466612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:05.466612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:05.466612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:05.466612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:05.466612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:05.467508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:05.467508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:05.467508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:05.467508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:05.467508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:05.467508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:05.467508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:05.467508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:05.467508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:05.467508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:05.467508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:05.467508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:05.467508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 237/483] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:34:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:05.474409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f29b0-2f59-4c20-badb-7d65cc7ab20a None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:05.598350 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f29b0-2f59-4c20-badb-7d65cc7ab20a None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:05.634492 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f483a-f8ee-451a-8507-37b91508b497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:05.635648 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f483a-f8ee-451a-8507-37b91508b497 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:05.636128 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f483a-f8ee-451a-8507-37b91508b497 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:05.636595 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f483a-f8ee-451a-8507-37b91508b497 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:05.637590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f483a-f8ee-451a-8507-37b91508b497 None admin] RBAC: Adding query filter params (name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:05.638092 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f483a-f8ee-451a-8507-37b91508b497 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:05.639667 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f483a-f8ee-451a-8507-37b91508b497 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:05.648496 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 247/484] 10.222.0.26 () {56 vars in 1103 bytes} [Mon May 9 09:34:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:05.654798 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcea9a-61cc-451c-bc83-9cabba73d6f5 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:05.789689 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1a293-327a-44f2-81a5-c0ec958171eb None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:05.790022 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1a293-327a-44f2-81a5-c0ec958171eb None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:05.790235 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1a293-327a-44f2-81a5-c0ec958171eb None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:05.793022 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 248/485] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:34:05.797193 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67160970-f38a-4802-b455-962fdc7fcfd3 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:05.797475 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67160970-f38a-4802-b455-962fdc7fcfd3 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:05.797751 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67160970-f38a-4802-b455-962fdc7fcfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:05.830283 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-67160970-f38a-4802-b455-962fdc7fcfd3 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:05.831496 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-67160970-f38a-4802-b455-962fdc7fcfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:05.847986 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcea9a-61cc-451c-bc83-9cabba73d6f5 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:05.919069 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188a700-997e-4a59-8b50-6809d28b6563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:05.919795 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188a700-997e-4a59-8b50-6809d28b6563 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:05.920075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188a700-997e-4a59-8b50-6809d28b6563 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:05.920402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188a700-997e-4a59-8b50-6809d28b6563 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:05.937362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8188a700-997e-4a59-8b50-6809d28b6563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:05.938844 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8188a700-997e-4a59-8b50-6809d28b6563 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:05.968168 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 249/486] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:05.970983 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-8188a700-997e-4a59-8b50-6809d28b6563 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:05.970983 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:05.970983 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:05.970983 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:05.970983 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:05.970983 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:05.970983 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:05.970983 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:05.970983 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:05.970983 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:05.970983 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:05.970983 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:05.970983 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:05.970983 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:05.970983 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:05.971834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:05.971834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:05.971834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:05.971834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:05.971834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:05.971834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:05.971834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:05.971834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:05.971834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:05.971834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:05.971834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:05.971834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:05.971834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:05.971834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:05.971834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:05.971834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:05.971834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:05.972691 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:05.972691 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:05.972691 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:05.972691 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:05.972691 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 238/487] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:05.979281 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67160970-f38a-4802-b455-962fdc7fcfd3 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:06.014849 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188a700-997e-4a59-8b50-6809d28b6563 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:06.110593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67160970-f38a-4802-b455-962fdc7fcfd3 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:06.138940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188a700-997e-4a59-8b50-6809d28b6563 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:06.145197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d494e3-6867-4be4-855d-ea5a5d15fea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:06.146396 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d494e3-6867-4be4-855d-ea5a5d15fea8 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:06.146868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d494e3-6867-4be4-855d-ea5a5d15fea8 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:06.147330 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d494e3-6867-4be4-855d-ea5a5d15fea8 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:06.148349 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d494e3-6867-4be4-855d-ea5a5d15fea8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:06.148890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d494e3-6867-4be4-855d-ea5a5d15fea8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:06.150564 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d494e3-6867-4be4-855d-ea5a5d15fea8 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:06.162460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 250/488] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:06.168281 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b116390-f1c2-4f4b-93ab-5b1dcaa8c2fc None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:06.169217 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b116390-f1c2-4f4b-93ab-5b1dcaa8c2fc None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:06.169842 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b116390-f1c2-4f4b-93ab-5b1dcaa8c2fc None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:06.171856 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d5186-9f64-49bb-971c-968de0c65cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:06.172294 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 251/489] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:06 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:06.172653 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d5186-9f64-49bb-971c-968de0c65cdf None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:06.172955 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d5186-9f64-49bb-971c-968de0c65cdf None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:06.173219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d5186-9f64-49bb-971c-968de0c65cdf None admin] PATH_INFO: `/v3/roles/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:06.174034 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b116390-f1c2-4f4b-93ab-5b1dcaa8c2fc None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:06.187760 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911d5186-9f64-49bb-971c-968de0c65cdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:06.195568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-911d5186-9f64-49bb-971c-968de0c65cdf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:06.195568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:06.195568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:06.195568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:06.195568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:06.195568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:06.195568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:06.195568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:06.195568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:06.195568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:06.195568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:06.195568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:06.195568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:06.195568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:06.195568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:06.195568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:06.196484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:06.196484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:06.196484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:06.196484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:06.196484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:06.196484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:06.196484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:06.196484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:06.196484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:06.196484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:06.196484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:06.196484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:06.196484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:06.196484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:06.196484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:06.196484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:06.197375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:06.197375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:06.197375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:06.197375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:06.197375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:06.197375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:06.197375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:06.197375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:06.197375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:06.197375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:06.197375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:06.197375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:06.197998 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 239/490] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:34:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:06.198881 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9418f7-acee-4da9-9903-8395a57b2635 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:06.199157 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9418f7-acee-4da9-9903-8395a57b2635 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:06.199429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9418f7-acee-4da9-9903-8395a57b2635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:06.217875 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-aa9418f7-acee-4da9-9903-8395a57b2635 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:06.219108 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-aa9418f7-acee-4da9-9903-8395a57b2635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:06.281672 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b116390-f1c2-4f4b-93ab-5b1dcaa8c2fc None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:06.287032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 240/491] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:06.287867 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9418f7-acee-4da9-9903-8395a57b2635 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:06.330139 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec97d9-d415-4e87-8747-cae4e4106360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:06.331815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec97d9-d415-4e87-8747-cae4e4106360 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:06.332275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec97d9-d415-4e87-8747-cae4e4106360 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:06.332747 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec97d9-d415-4e87-8747-cae4e4106360 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:06.341959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec97d9-d415-4e87-8747-cae4e4106360 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:06.344337 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec97d9-d415-4e87-8747-cae4e4106360 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:06.363202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-88ec97d9-d415-4e87-8747-cae4e4106360 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 09:34:06.363202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:06.363202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:06.363202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:06.363202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:06.363202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:06.363202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:06.363202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:06.363202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:06.363202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:06.363202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:06.363202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:06.363202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:06.363202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:06.363202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:06.363202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:06.364208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:06.364208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:06.364208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:06.364208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:06.364208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:06.364208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:06.364208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:06.364208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:06.364208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:06.364208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:06.364208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:06.364208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:06.364208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:06.364208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:06.364208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:06.364208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:06.364208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:06.364208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:06.365553 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:06.365553 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:06.365553 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:06.365553 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:06.365553 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:06.365553 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:06.365553 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:06.365553 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:06.365553 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 09:34:06.365553 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:06.365553 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 252/492] 10.222.0.26 () {56 vars in 1093 bytes} [Mon May 9 09:34:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:06.366152 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e712e-4947-4977-b783-114e92556cc2 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:06.366152 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e712e-4947-4977-b783-114e92556cc2 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:06.366271 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e712e-4947-4977-b783-114e92556cc2 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:06.367454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 253/493] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:06.372023 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e712e-4947-4977-b783-114e92556cc2 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:06.392883 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9418f7-acee-4da9-9903-8395a57b2635 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:06.425472 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db7479-da71-43c4-ba52-6c16704f9cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:06.426057 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db7479-da71-43c4-ba52-6c16704f9cd9 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:06.426335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db7479-da71-43c4-ba52-6c16704f9cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:06.426607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db7479-da71-43c4-ba52-6c16704f9cd9 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:06.427351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db7479-da71-43c4-ba52-6c16704f9cd9 None admin] RBAC: Adding query filter params (name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:06.427667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db7479-da71-43c4-ba52-6c16704f9cd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:06.429079 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db7479-da71-43c4-ba52-6c16704f9cd9 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:06.437128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 241/494] 10.222.0.26 () {56 vars in 1103 bytes} [Mon May 9 09:34:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:06.438005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db7479-da71-43c4-ba52-6c16704f9cd9 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:06.479502 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e712e-4947-4977-b783-114e92556cc2 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:06.514717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2422b-d545-4cb4-b2f3-5009bd17d955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:06.516029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2422b-d545-4cb4-b2f3-5009bd17d955 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:06.516322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2422b-d545-4cb4-b2f3-5009bd17d955 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:06.516593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2422b-d545-4cb4-b2f3-5009bd17d955 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:06.527322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2422b-d545-4cb4-b2f3-5009bd17d955 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:06.532963 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db7479-da71-43c4-ba52-6c16704f9cd9 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:06.535640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-1ef2422b-d545-4cb4-b2f3-5009bd17d955 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:06.535640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:06.535640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:06.535640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:06.535640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:06.535640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:06.535640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:06.535640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:06.535640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:06.535640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:06.535640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:06.535640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:06.535640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:06.535640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:06.535640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:06.535640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:06.536543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:06.536543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:06.536543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:06.536543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:06.536543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:06.536543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:06.536543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:06.536543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:06.536543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:06.536543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:06.536543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:06.536543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:06.536543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:06.536543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:06.536543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:06.536543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:06.537669 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:06.537669 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:06.537669 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:06.537669 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:06.537669 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:06.537669 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:06.537669 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:06.537669 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:06.537669 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:06.537669 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:06.537669 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:06.537669 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:06.537669 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 254/495] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:34:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:06.540615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56836daf-e734-45ec-867f-4a99549ba664 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:06.541028 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56836daf-e734-45ec-867f-4a99549ba664 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:06.541291 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56836daf-e734-45ec-867f-4a99549ba664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:06.557566 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-56836daf-e734-45ec-867f-4a99549ba664 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:06.558827 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-56836daf-e734-45ec-867f-4a99549ba664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:06.572941 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59108fdd-deb6-4635-829d-abfa9fe3a9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:06.573695 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59108fdd-deb6-4635-829d-abfa9fe3a9de None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:06.573978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59108fdd-deb6-4635-829d-abfa9fe3a9de None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:06.574259 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59108fdd-deb6-4635-829d-abfa9fe3a9de None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:06.575096 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59108fdd-deb6-4635-829d-abfa9fe3a9de None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:06.575403 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59108fdd-deb6-4635-829d-abfa9fe3a9de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:06.576861 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59108fdd-deb6-4635-829d-abfa9fe3a9de None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:06.619630 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 242/496] 10.222.0.26 () {56 vars in 1133 bytes} [Mon May 9 09:34:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:06.619630 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59108fdd-deb6-4635-829d-abfa9fe3a9de None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:06.640714 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 255/497] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:06.640714 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56836daf-e734-45ec-867f-4a99549ba664 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:06.735253 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56836daf-e734-45ec-867f-4a99549ba664 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:06.735873 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59108fdd-deb6-4635-829d-abfa9fe3a9de None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:06.768853 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d67fe-82ca-4c90-bf8f-956f52894b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:06.769770 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d67fe-82ca-4c90-bf8f-956f52894b8f None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:06.770044 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d67fe-82ca-4c90-bf8f-956f52894b8f None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:06.770314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d67fe-82ca-4c90-bf8f-956f52894b8f None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:06.771136 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d67fe-82ca-4c90-bf8f-956f52894b8f None admin] RBAC: Adding query filter params (name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:06.771424 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d67fe-82ca-4c90-bf8f-956f52894b8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:06.772892 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d67fe-82ca-4c90-bf8f-956f52894b8f None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:06.773135 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacefbd-d2cc-48b9-8741-1c21648c6dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:06.773987 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacefbd-d2cc-48b9-8741-1c21648c6dd1 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:06.774264 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacefbd-d2cc-48b9-8741-1c21648c6dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:06.774528 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacefbd-d2cc-48b9-8741-1c21648c6dd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:06.781071 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 256/498] 10.222.0.26 () {56 vars in 1103 bytes} [Mon May 9 09:34:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:06.781699 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d67fe-82ca-4c90-bf8f-956f52894b8f None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:06.788604 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bacefbd-d2cc-48b9-8741-1c21648c6dd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:06.790118 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bacefbd-d2cc-48b9-8741-1c21648c6dd1 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:06.807595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-4bacefbd-d2cc-48b9-8741-1c21648c6dd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:06.807595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:06.807595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:06.807595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:06.807595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:06.807595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:06.807595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:06.807595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:06.807595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:06.807595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:06.807595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:06.807595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:06.807595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:06.807595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:06.807595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:06.808574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:06.808574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:06.808574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:06.808574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:06.808574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:06.808574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:06.808574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:06.808574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:06.808574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:06.808574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:06.808574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:06.808574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:06.808574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:06.808574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:06.808574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:06.808574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:06.808574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:06.809581 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:06.809581 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:06.809581 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:06.809581 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:06.810150 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 243/499] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:06.811084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacefbd-d2cc-48b9-8741-1c21648c6dd1 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:06.871889 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d67fe-82ca-4c90-bf8f-956f52894b8f None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:06.903310 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacefbd-d2cc-48b9-8741-1c21648c6dd1 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:06.903874 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bfbf47-cfe7-491a-b968-669d7afb0aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:06.904511 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfbf47-cfe7-491a-b968-669d7afb0aaf None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:06.904789 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfbf47-cfe7-491a-b968-669d7afb0aaf None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:06.905057 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfbf47-cfe7-491a-b968-669d7afb0aaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:06.935465 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8f78e-922d-4e6f-bf05-3fb3ce4446d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:06.936238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8f78e-922d-4e6f-bf05-3fb3ce4446d9 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:06.936512 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8f78e-922d-4e6f-bf05-3fb3ce4446d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:06.936814 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8f78e-922d-4e6f-bf05-3fb3ce4446d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:06.946474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e8f78e-922d-4e6f-bf05-3fb3ce4446d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:06.954216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-51e8f78e-922d-4e6f-bf05-3fb3ce4446d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:06.954216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:06.954216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:06.954216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:06.954216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:06.954216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:06.954216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:06.954216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:06.954216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:06.954216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:06.954216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:06.954216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:06.954216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:06.954216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:06.954216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:06.954216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:06.955144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:06.955144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:06.955144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:06.955144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:06.955144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:06.955144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:06.955144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:06.955144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:06.955144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:06.955144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:06.955144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:06.955144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:06.955144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:06.955144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:06.955144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:06.955144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:06.955968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:06.955968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:06.955968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:06.955968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:06.955968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:06.955968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:06.955968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:06.955968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:06.955968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:06.955968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:06.955968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:06.955968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:06.955968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 244/500] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:34:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:06.956655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8f78e-922d-4e6f-bf05-3fb3ce4446d9 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:06.970246 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bfbf47-cfe7-491a-b968-669d7afb0aaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:06.971696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bfbf47-cfe7-491a-b968-669d7afb0aaf None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:07.050611 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8f78e-922d-4e6f-bf05-3fb3ce4446d9 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:07.064807 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 257/501] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:34:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:34:07.066253 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ece7a-107e-45d2-b4b9-01f98e23bb2e None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:07.066603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ece7a-107e-45d2-b4b9-01f98e23bb2e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:07.066940 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ece7a-107e-45d2-b4b9-01f98e23bb2e None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:07.068263 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 258/502] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:07.069411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ece7a-107e-45d2-b4b9-01f98e23bb2e None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:07.086279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e6663-b53d-42f5-a43b-108ac4c69d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:07.087141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e6663-b53d-42f5-a43b-108ac4c69d76 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:07.087421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e6663-b53d-42f5-a43b-108ac4c69d76 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:07.087689 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e6663-b53d-42f5-a43b-108ac4c69d76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:07.097783 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e6663-b53d-42f5-a43b-108ac4c69d76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:07.099265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e6663-b53d-42f5-a43b-108ac4c69d76 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:07.115521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-418e6663-b53d-42f5-a43b-108ac4c69d76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:07.115521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:07.115521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:07.115521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:07.115521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:07.115521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:07.115521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:07.115521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:07.115521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:07.115521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:07.115521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:07.115521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:07.115521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:07.115521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:07.115521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:07.116447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:07.116447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:07.116447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:07.116447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:07.116447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:07.116447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:07.116447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:07.116447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:07.116447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:07.116447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:07.116447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:07.116447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:07.116447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:07.116447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:07.116447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:07.116447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:07.116447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:07.117344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:07.117344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:07.117344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:07.117344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:07.117344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 245/503] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:07.117610 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e6663-b53d-42f5-a43b-108ac4c69d76 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:07.166468 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ece7a-107e-45d2-b4b9-01f98e23bb2e None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:07.200070 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce151269-9ddb-46fd-9aec-8f1fa9e5cb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:07.200750 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce151269-9ddb-46fd-9aec-8f1fa9e5cb93 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:07.201031 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce151269-9ddb-46fd-9aec-8f1fa9e5cb93 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:07.201304 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce151269-9ddb-46fd-9aec-8f1fa9e5cb93 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:07.202025 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce151269-9ddb-46fd-9aec-8f1fa9e5cb93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:07.202329 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce151269-9ddb-46fd-9aec-8f1fa9e5cb93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:07.203724 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce151269-9ddb-46fd-9aec-8f1fa9e5cb93 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:07.211631 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e6663-b53d-42f5-a43b-108ac4c69d76 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:07.214821 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 259/504] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:07.215608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce151269-9ddb-46fd-9aec-8f1fa9e5cb93 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:07.244107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd615e-bb91-422c-9534-7d5903cd7fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:07.244849 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd615e-bb91-422c-9534-7d5903cd7fe1 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:07.245125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd615e-bb91-422c-9534-7d5903cd7fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:07.245387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd615e-bb91-422c-9534-7d5903cd7fe1 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:07.246451 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd615e-bb91-422c-9534-7d5903cd7fe1 None admin] RBAC: Adding query filter params (name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:07.246751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd615e-bb91-422c-9534-7d5903cd7fe1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:07.248141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd615e-bb91-422c-9534-7d5903cd7fe1 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:07.255762 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 246/505] 10.222.0.26 () {56 vars in 1103 bytes} [Mon May 9 09:34:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:07.256933 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6af01-e5e2-4b80-a85b-459d97e2e8b1 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:07.257210 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6af01-e5e2-4b80-a85b-459d97e2e8b1 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:07.257481 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6af01-e5e2-4b80-a85b-459d97e2e8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:07.273303 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-15c6af01-e5e2-4b80-a85b-459d97e2e8b1 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:07.274483 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-15c6af01-e5e2-4b80-a85b-459d97e2e8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:07.310992 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce151269-9ddb-46fd-9aec-8f1fa9e5cb93 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:07.341215 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 247/506] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:07.342055 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6af01-e5e2-4b80-a85b-459d97e2e8b1 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:07.353549 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88769dd2-92c6-4232-8563-2937f20b65cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:07.354362 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88769dd2-92c6-4232-8563-2937f20b65cf None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:07.354636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88769dd2-92c6-4232-8563-2937f20b65cf None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:07.354902 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88769dd2-92c6-4232-8563-2937f20b65cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:07.365601 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88769dd2-92c6-4232-8563-2937f20b65cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:07.367050 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88769dd2-92c6-4232-8563-2937f20b65cf None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:07.384208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-88769dd2-92c6-4232-8563-2937f20b65cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:07.384208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:07.384208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:07.384208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:07.384208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:07.384208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:07.384208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:07.384208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:07.384208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:07.384208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:07.384208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:07.384208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:07.384208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:07.384208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:07.384208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:07.385183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:07.385183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:07.385183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:07.385183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:07.385183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:07.385183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:07.385183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:07.385183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:07.385183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:07.385183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:07.385183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:07.385183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:07.385183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:07.385183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:07.385183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:07.385183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:07.385183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:07.386061 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:07.386061 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:07.386061 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:07.386061 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:07.386061 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 260/507] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:07.386346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88769dd2-92c6-4232-8563-2937f20b65cf None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:07.442427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6af01-e5e2-4b80-a85b-459d97e2e8b1 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:07.473807 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227cb30-6e7c-450e-a034-ddf8a207da57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:07.474462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227cb30-6e7c-450e-a034-ddf8a207da57 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:07.474739 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227cb30-6e7c-450e-a034-ddf8a207da57 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:07.475008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227cb30-6e7c-450e-a034-ddf8a207da57 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:07.475696 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227cb30-6e7c-450e-a034-ddf8a207da57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:07.475998 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227cb30-6e7c-450e-a034-ddf8a207da57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:07.477398 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227cb30-6e7c-450e-a034-ddf8a207da57 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:07.480972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88769dd2-92c6-4232-8563-2937f20b65cf None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:07.488113 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 248/508] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:07.488930 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227cb30-6e7c-450e-a034-ddf8a207da57 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:07.513134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a1d83b-1238-4868-8a6b-eede5b0800d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:07.513914 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a1d83b-1238-4868-8a6b-eede5b0800d1 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:07.514189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a1d83b-1238-4868-8a6b-eede5b0800d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:07.514481 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a1d83b-1238-4868-8a6b-eede5b0800d1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:07.522054 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a1d83b-1238-4868-8a6b-eede5b0800d1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:07.523492 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a1d83b-1238-4868-8a6b-eede5b0800d1 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:07.537011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-12a1d83b-1238-4868-8a6b-eede5b0800d1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 09:34:07.537011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:07.537011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:07.537011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:07.537011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:07.537011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:07.537011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:07.537011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:07.537011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:07.537011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:07.537011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:07.537011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:07.537011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:07.537011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:07.537011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:07.537011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:07.537929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:07.537929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:07.537929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:07.537929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:07.537929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:07.537929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:07.537929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:07.537929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:07.537929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:07.537929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:07.537929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:07.537929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:07.537929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:07.537929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:07.537929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:07.537929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:07.537929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:07.537929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:07.538858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:07.538858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:07.538858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:07.538858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:07.538858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:07.538858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:07.538858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:07.538858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:07.538858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 09:34:07.538858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:07.540510 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 261/509] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:34:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:07.545699 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a1d83b-1238-4868-8a6b-eede5b0800d1 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:07.587746 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227cb30-6e7c-450e-a034-ddf8a207da57 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:07.619988 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28063254-87a5-4733-baf6-e60c36bf0088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:07.621395 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28063254-87a5-4733-baf6-e60c36bf0088 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:07.621667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28063254-87a5-4733-baf6-e60c36bf0088 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:07.621926 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28063254-87a5-4733-baf6-e60c36bf0088 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:07.632071 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28063254-87a5-4733-baf6-e60c36bf0088 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:07.633543 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28063254-87a5-4733-baf6-e60c36bf0088 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:07.639983 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a1d83b-1238-4868-8a6b-eede5b0800d1 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:07.650104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-28063254-87a5-4733-baf6-e60c36bf0088 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:07.650104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:07.650104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:07.650104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:07.650104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:07.650104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:07.650104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:07.650104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:07.650104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:07.650104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:07.650104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:07.650104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:07.650104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:07.650104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:07.650104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:07.650990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:07.650990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:07.650990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:07.650990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:07.650990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:07.650990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:07.650990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:07.650990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:07.650990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:07.650990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:07.650990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:07.650990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:07.650990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:07.650990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:07.650990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:07.650990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:07.650990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:07.651868 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:07.651868 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:07.651868 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:07.651868 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:07.651868 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 249/510] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:07.652123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28063254-87a5-4733-baf6-e60c36bf0088 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:07.672951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc066af-0b9b-4cb4-b4f3-6935338b0006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:07.673746 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc066af-0b9b-4cb4-b4f3-6935338b0006 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:07.674028 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc066af-0b9b-4cb4-b4f3-6935338b0006 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:07.674300 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc066af-0b9b-4cb4-b4f3-6935338b0006 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:07.684825 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc066af-0b9b-4cb4-b4f3-6935338b0006 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:07.686295 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc066af-0b9b-4cb4-b4f3-6935338b0006 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:07.702832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-4fc066af-0b9b-4cb4-b4f3-6935338b0006 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:07.702832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:07.702832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:07.702832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:07.702832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:07.702832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:07.702832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:07.702832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:07.702832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:07.702832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:07.702832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:07.702832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:07.702832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:07.702832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:07.702832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:07.703696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:07.703696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:07.703696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:07.703696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:07.703696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:07.703696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:07.703696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:07.703696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:07.703696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:07.703696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:07.703696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:07.703696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:07.703696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:07.703696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:07.703696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:07.703696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:07.703696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:07.704880 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:07.704880 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:07.704880 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:07.704880 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:07.704880 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 262/511] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:07.704880 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc066af-0b9b-4cb4-b4f3-6935338b0006 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:07.759114 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28063254-87a5-4733-baf6-e60c36bf0088 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:07.791864 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f7c65-4766-4499-8752-8a5270fd878a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:07.792533 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f7c65-4766-4499-8752-8a5270fd878a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:07.792841 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f7c65-4766-4499-8752-8a5270fd878a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:07.793102 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f7c65-4766-4499-8752-8a5270fd878a None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:07.793810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f7c65-4766-4499-8752-8a5270fd878a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:07.794149 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f7c65-4766-4499-8752-8a5270fd878a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:07.795528 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f7c65-4766-4499-8752-8a5270fd878a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:07.796783 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc066af-0b9b-4cb4-b4f3-6935338b0006 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:07.806819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 250/512] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:07.807589 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f7c65-4766-4499-8752-8a5270fd878a None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:07.830357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621aa02e-7900-40f4-a598-ec6c3e9bc44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:07.831240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621aa02e-7900-40f4-a598-ec6c3e9bc44a None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:07.831518 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621aa02e-7900-40f4-a598-ec6c3e9bc44a None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:07.831780 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621aa02e-7900-40f4-a598-ec6c3e9bc44a None admin] PATH_INFO: `/v3/users/glance` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:07.838854 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621aa02e-7900-40f4-a598-ec6c3e9bc44a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:07.840265 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621aa02e-7900-40f4-a598-ec6c3e9bc44a None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:07.854558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-621aa02e-7900-40f4-a598-ec6c3e9bc44a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 09:34:07.854558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:07.854558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:07.854558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:07.854558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:07.854558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:07.854558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:07.854558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:07.854558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:07.854558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:07.854558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:07.854558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:07.854558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:07.854558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:07.854558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:07.854558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:07.855567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:07.855567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:07.855567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:07.855567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:07.855567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:07.855567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:07.855567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:07.855567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:07.855567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:07.855567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:07.855567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:07.855567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:07.855567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:07.855567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:07.855567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:07.855567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:07.855567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:07.855567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:07.856467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:07.856467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:07.856467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:07.856467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:07.856467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:07.856467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:07.856467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:07.856467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:07.856467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 09:34:07.856467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:07.856467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 263/513] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:34:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:07.856467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621aa02e-7900-40f4-a598-ec6c3e9bc44a None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:07.907419 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f7c65-4766-4499-8752-8a5270fd878a None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:07.940683 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5d246-eac2-43b3-8672-1808d16d18a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:07.941446 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5d246-eac2-43b3-8672-1808d16d18a4 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:07.941786 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5d246-eac2-43b3-8672-1808d16d18a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:07.942062 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5d246-eac2-43b3-8672-1808d16d18a4 None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:07.942898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5d246-eac2-43b3-8672-1808d16d18a4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:07.943204 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5d246-eac2-43b3-8672-1808d16d18a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:07.944654 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5d246-eac2-43b3-8672-1808d16d18a4 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:07.949976 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621aa02e-7900-40f4-a598-ec6c3e9bc44a None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:07.960967 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 251/514] 10.222.0.26 () {56 vars in 1139 bytes} [Mon May 9 09:34:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:07.961761 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5d246-eac2-43b3-8672-1808d16d18a4 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:07.982185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d8555-c91d-4cb3-99df-ead287da3ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:07.982837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d8555-c91d-4cb3-99df-ead287da3ebf None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:07.983110 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d8555-c91d-4cb3-99df-ead287da3ebf None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:07.983376 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d8555-c91d-4cb3-99df-ead287da3ebf None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:07.984087 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d8555-c91d-4cb3-99df-ead287da3ebf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:07.984384 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d8555-c91d-4cb3-99df-ead287da3ebf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:07.985794 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d8555-c91d-4cb3-99df-ead287da3ebf None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:07.997090 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 264/515] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:07.997918 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d8555-c91d-4cb3-99df-ead287da3ebf None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:08.059017 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5d246-eac2-43b3-8672-1808d16d18a4 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:08.092489 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e43cc5-2d08-4079-afe5-c2de4e826dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:08.093131 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e43cc5-2d08-4079-afe5-c2de4e826dab None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:08.093409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e43cc5-2d08-4079-afe5-c2de4e826dab None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:08.093677 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e43cc5-2d08-4079-afe5-c2de4e826dab None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:08.094356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d8555-c91d-4cb3-99df-ead287da3ebf None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:08.094458 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e43cc5-2d08-4079-afe5-c2de4e826dab None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:08.094667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e43cc5-2d08-4079-afe5-c2de4e826dab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:08.096119 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e43cc5-2d08-4079-afe5-c2de4e826dab None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:08.106819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 252/516] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:08.107609 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e43cc5-2d08-4079-afe5-c2de4e826dab None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:08.128062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961b9bb-c262-4ea5-a313-3294e857c0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:08.128880 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961b9bb-c262-4ea5-a313-3294e857c0f9 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:08.129160 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961b9bb-c262-4ea5-a313-3294e857c0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:08.129428 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961b9bb-c262-4ea5-a313-3294e857c0f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:08.135846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961b9bb-c262-4ea5-a313-3294e857c0f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:08.137321 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961b9bb-c262-4ea5-a313-3294e857c0f9 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:08.151598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-d961b9bb-c262-4ea5-a313-3294e857c0f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:08.151598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:08.151598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:08.151598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:08.151598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:08.151598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:08.151598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:08.151598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:08.151598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:08.151598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:08.151598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:08.151598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:08.151598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:08.151598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:08.151598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:08.152442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:08.152442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:08.152442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:08.152442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:08.152442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:08.152442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:08.152442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:08.152442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:08.152442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:08.152442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:08.152442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:08.152442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:08.152442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:08.152442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:08.152442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:08.152442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:08.152442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:08.153377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:08.153377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:08.153377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:08.153377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:08.153377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:08.153377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:08.153377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:08.153377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:08.153377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 265/517] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:08.153854 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961b9bb-c262-4ea5-a313-3294e857c0f9 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:08.199831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e43cc5-2d08-4079-afe5-c2de4e826dab None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:08.231100 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539588ae-a825-4a33-84aa-98d1cf301f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:08.231756 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539588ae-a825-4a33-84aa-98d1cf301f55 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:08.232033 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539588ae-a825-4a33-84aa-98d1cf301f55 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:08.232299 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539588ae-a825-4a33-84aa-98d1cf301f55 None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:08.233157 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539588ae-a825-4a33-84aa-98d1cf301f55 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:08.233461 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539588ae-a825-4a33-84aa-98d1cf301f55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:08.234875 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539588ae-a825-4a33-84aa-98d1cf301f55 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:08.245250 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961b9bb-c262-4ea5-a313-3294e857c0f9 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:08.252014 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 253/518] 10.222.0.26 () {56 vars in 1137 bytes} [Mon May 9 09:34:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:08.252811 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539588ae-a825-4a33-84aa-98d1cf301f55 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:08.281496 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d473d8e-a920-4db4-a9d4-69f0d9ea6b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:08.282402 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d473d8e-a920-4db4-a9d4-69f0d9ea6b87 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:08.282778 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d473d8e-a920-4db4-a9d4-69f0d9ea6b87 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:08.283246 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d473d8e-a920-4db4-a9d4-69f0d9ea6b87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:08.347018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d473d8e-a920-4db4-a9d4-69f0d9ea6b87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:08.348413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d473d8e-a920-4db4-a9d4-69f0d9ea6b87 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:08.351097 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539588ae-a825-4a33-84aa-98d1cf301f55 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:08.383496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc57de-7f33-412a-b234-b091dfdf374d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:08.384282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc57de-7f33-412a-b234-b091dfdf374d None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:08.384558 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc57de-7f33-412a-b234-b091dfdf374d None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:08.384897 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc57de-7f33-412a-b234-b091dfdf374d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:08.392082 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc57de-7f33-412a-b234-b091dfdf374d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:08.393561 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc57de-7f33-412a-b234-b091dfdf374d None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:08.409188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-5afc57de-7f33-412a-b234-b091dfdf374d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 09:34:08.409188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:08.409188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:08.409188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:08.409188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:08.409188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:08.409188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:08.409188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:08.409188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:08.409188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:08.409188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:08.409188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:08.409188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:08.409188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:08.409188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:08.409188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:08.410169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:08.410169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:08.410169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:08.410169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:08.410169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:08.410169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:08.410169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:08.410169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:08.410169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:08.410169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:08.410169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:08.410169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:08.410169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:08.410169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:08.410169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:08.410169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:08.410169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:08.410169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:08.411077 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:08.411077 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:08.411077 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:08.411077 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:08.411077 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:08.411077 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:08.411077 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:08.411077 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:08.411077 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 09:34:08.411077 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:08.411077 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 254/519] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:34:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:08.411640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc57de-7f33-412a-b234-b091dfdf374d None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:08.440239 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 266/520] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:34:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:34:08.441087 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d473d8e-a920-4db4-a9d4-69f0d9ea6b87 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:08.507211 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc57de-7f33-412a-b234-b091dfdf374d None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:08.537317 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d473d8e-a920-4db4-a9d4-69f0d9ea6b87 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:08.542648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522abfb1-5050-42f9-bf71-bc5d9027af08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:08.543315 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522abfb1-5050-42f9-bf71-bc5d9027af08 None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:08.543588 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522abfb1-5050-42f9-bf71-bc5d9027af08 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:08.543856 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522abfb1-5050-42f9-bf71-bc5d9027af08 None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:08.544560 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522abfb1-5050-42f9-bf71-bc5d9027af08 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:08.546046 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522abfb1-5050-42f9-bf71-bc5d9027af08 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:08.567431 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 255/521] 10.222.0.26 () {60 vars in 1132 bytes} [Mon May 9 09:34:08 2022] POST /identity/v3/users => generated 247 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:34:08.568261 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522abfb1-5050-42f9-bf71-bc5d9027af08 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:08.576187 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb2373-d26e-4d82-899b-98d5e5cf6339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:08.576902 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb2373-d26e-4d82-899b-98d5e5cf6339 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:08.577196 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb2373-d26e-4d82-899b-98d5e5cf6339 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:08.577464 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb2373-d26e-4d82-899b-98d5e5cf6339 None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:08.578278 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb2373-d26e-4d82-899b-98d5e5cf6339 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:08.578586 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb2373-d26e-4d82-899b-98d5e5cf6339 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:08.580084 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb2373-d26e-4d82-899b-98d5e5cf6339 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:08.599851 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 267/522] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:08.601082 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb2373-d26e-4d82-899b-98d5e5cf6339 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:08.677803 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522abfb1-5050-42f9-bf71-bc5d9027af08 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:08.705420 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb2373-d26e-4d82-899b-98d5e5cf6339 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:08.713151 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f21a7-b7b5-4be2-a070-bb5f38e9f862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:08.713923 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f21a7-b7b5-4be2-a070-bb5f38e9f862 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:08.714210 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f21a7-b7b5-4be2-a070-bb5f38e9f862 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:08.714480 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f21a7-b7b5-4be2-a070-bb5f38e9f862 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:08.739075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f6e53-e108-4133-9879-0dec36d0eea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:08.739790 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f6e53-e108-4133-9879-0dec36d0eea6 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:08.740040 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f6e53-e108-4133-9879-0dec36d0eea6 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:08.740336 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f6e53-e108-4133-9879-0dec36d0eea6 None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:08.741225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f6e53-e108-4133-9879-0dec36d0eea6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:08.741520 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f6e53-e108-4133-9879-0dec36d0eea6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:08.742965 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f6e53-e108-4133-9879-0dec36d0eea6 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:08.779530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 268/523] 10.222.0.26 () {56 vars in 1137 bytes} [Mon May 9 09:34:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:08.780451 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f6e53-e108-4133-9879-0dec36d0eea6 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:08.785864 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f21a7-b7b5-4be2-a070-bb5f38e9f862 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:08.788220 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f21a7-b7b5-4be2-a070-bb5f38e9f862 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:08.875835 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f6e53-e108-4133-9879-0dec36d0eea6 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:08.883064 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 256/524] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:34:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:34:08.884169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f21a7-b7b5-4be2-a070-bb5f38e9f862 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:08.908121 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b700d-8aac-4c84-8ac9-df06905598e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:08.908957 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b700d-8aac-4c84-8ac9-df06905598e4 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:08.909235 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b700d-8aac-4c84-8ac9-df06905598e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:08.909590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b700d-8aac-4c84-8ac9-df06905598e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:08.920133 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b700d-8aac-4c84-8ac9-df06905598e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:08.921638 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b700d-8aac-4c84-8ac9-df06905598e4 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:08.938474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-087b700d-8aac-4c84-8ac9-df06905598e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:08.938474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:08.938474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:08.938474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:08.938474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:08.938474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:08.938474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:08.938474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:08.938474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:08.938474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:08.938474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:08.938474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:08.938474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:08.938474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:08.938474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:08.939340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:08.939340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:08.939340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:08.939340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:08.939340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:08.939340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:08.939340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:08.939340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:08.939340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:08.939340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:08.939340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:08.939340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:08.939340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:08.939340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:08.939340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:08.939340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:08.939340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:08.940191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:08.940191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:08.940191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:08.940191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:08.940191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 269/525] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:08.940992 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b700d-8aac-4c84-8ac9-df06905598e4 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:08.980997 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f21a7-b7b5-4be2-a070-bb5f38e9f862 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:09.014873 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b867565-4db6-41ed-a323-5d2e169dc35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:09.015520 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b867565-4db6-41ed-a323-5d2e169dc35e None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:09.015795 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b867565-4db6-41ed-a323-5d2e169dc35e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:09.016058 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b867565-4db6-41ed-a323-5d2e169dc35e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:09.016911 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b867565-4db6-41ed-a323-5d2e169dc35e None admin] RBAC: Adding query filter params (role.id=f55c3728df78466989d0cd1b13c5ea28, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=6362303cb918486c8525225914332ad5) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:09.017438 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b867565-4db6-41ed-a323-5d2e169dc35e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:09.018877 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b867565-4db6-41ed-a323-5d2e169dc35e None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:09.024818 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 257/526] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:34:08 2022] GET /identity/v3/role_assignments?user.id=6362303cb918486c8525225914332ad5&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=f55c3728df78466989d0cd1b13c5ea28 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:09.025694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b867565-4db6-41ed-a323-5d2e169dc35e None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:09.039899 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b700d-8aac-4c84-8ac9-df06905598e4 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:09.077043 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365fcf3-6b93-443e-8784-b1f1f387085f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:09.078012 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365fcf3-6b93-443e-8784-b1f1f387085f None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:09.078494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365fcf3-6b93-443e-8784-b1f1f387085f None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:09.078943 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365fcf3-6b93-443e-8784-b1f1f387085f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:09.140388 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b867565-4db6-41ed-a323-5d2e169dc35e None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:09.156129 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365fcf3-6b93-443e-8784-b1f1f387085f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:09.158809 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365fcf3-6b93-443e-8784-b1f1f387085f None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:09.176684 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdccfe-934d-4bb5-bb5c-d1ff1fbf8a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:09.177488 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdccfe-934d-4bb5-bb5c-d1ff1fbf8a4c None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:09.177769 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdccfe-934d-4bb5-bb5c-d1ff1fbf8a4c None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:09.178042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdccfe-934d-4bb5-bb5c-d1ff1fbf8a4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:09.189066 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdccfe-934d-4bb5-bb5c-d1ff1fbf8a4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:09.190647 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdccfe-934d-4bb5-bb5c-d1ff1fbf8a4c None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:09.209475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-66fdccfe-934d-4bb5-bb5c-d1ff1fbf8a4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:09.209475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:09.209475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:09.209475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:09.209475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:09.209475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:09.209475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:09.209475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:09.209475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:09.209475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:09.209475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:09.209475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:09.209475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:09.209475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:09.209475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:09.210400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:09.210400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:09.210400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:09.210400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:09.210400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:09.210400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:09.210400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:09.210400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:09.210400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:09.210400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:09.210400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:09.210400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:09.210400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:09.210400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:09.210400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:09.210400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:09.210400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:09.211285 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:09.211285 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:09.211285 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:09.211285 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:09.211515 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 258/527] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:09.212252 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdccfe-934d-4bb5-bb5c-d1ff1fbf8a4c None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:09.265038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 270/528] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:34:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:34:09.266569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365fcf3-6b93-443e-8784-b1f1f387085f None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:09.312128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdccfe-934d-4bb5-bb5c-d1ff1fbf8a4c None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:09.350373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446dad4-6db6-426b-b3f8-13195def8f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:09.351065 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446dad4-6db6-426b-b3f8-13195def8f04 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:09.351347 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446dad4-6db6-426b-b3f8-13195def8f04 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:09.351619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446dad4-6db6-426b-b3f8-13195def8f04 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:09.352348 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446dad4-6db6-426b-b3f8-13195def8f04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:09.352649 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446dad4-6db6-426b-b3f8-13195def8f04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:09.354302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446dad4-6db6-426b-b3f8-13195def8f04 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:09.365861 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 259/529] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:09.366709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446dad4-6db6-426b-b3f8-13195def8f04 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:09.368212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365fcf3-6b93-443e-8784-b1f1f387085f None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:09.406637 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b0440-e3ed-424a-a947-810fd914475c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:09.407636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b0440-e3ed-424a-a947-810fd914475c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:09.408111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b0440-e3ed-424a-a947-810fd914475c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:09.408574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b0440-e3ed-424a-a947-810fd914475c None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:09.409535 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b0440-e3ed-424a-a947-810fd914475c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:09.410031 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b0440-e3ed-424a-a947-810fd914475c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:09.411633 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b0440-e3ed-424a-a947-810fd914475c None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:09.423661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 271/530] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:09.424886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b0440-e3ed-424a-a947-810fd914475c None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:09.473421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446dad4-6db6-426b-b3f8-13195def8f04 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:09.506488 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e00e3-5683-487e-8506-a3331184da06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:09.507276 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e00e3-5683-487e-8506-a3331184da06 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:09.507552 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e00e3-5683-487e-8506-a3331184da06 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:09.507818 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e00e3-5683-487e-8506-a3331184da06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:09.519312 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e00e3-5683-487e-8506-a3331184da06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:09.520844 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e00e3-5683-487e-8506-a3331184da06 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:09.524145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b0440-e3ed-424a-a947-810fd914475c None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:09.538344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-554e00e3-5683-487e-8506-a3331184da06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:09.538344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:09.538344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:09.538344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:09.538344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:09.538344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:09.538344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:09.538344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:09.538344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:09.538344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:09.538344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:09.538344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:09.538344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:09.538344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:09.538344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:09.539306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:09.539306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:09.539306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:09.539306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:09.539306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:09.539306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:09.539306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:09.539306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:09.539306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:09.539306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:09.539306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:09.539306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:09.539306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:09.539306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:09.539306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:09.539306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:09.539306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:09.540136 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:09.540136 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:09.540136 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:09.540136 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:09.541500 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 260/531] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:09.542333 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e00e3-5683-487e-8506-a3331184da06 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:09.561140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1bf49-e6d4-47d5-a882-b8688ee0ad20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:09.562925 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1bf49-e6d4-47d5-a882-b8688ee0ad20 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:09.563406 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1bf49-e6d4-47d5-a882-b8688ee0ad20 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:09.563876 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1bf49-e6d4-47d5-a882-b8688ee0ad20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:09.571018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c1bf49-e6d4-47d5-a882-b8688ee0ad20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:09.572782 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c1bf49-e6d4-47d5-a882-b8688ee0ad20 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:09.587173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-a4c1bf49-e6d4-47d5-a882-b8688ee0ad20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:09.587173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:09.587173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:09.587173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:09.587173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:09.587173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:09.587173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:09.587173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:09.587173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:09.587173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:09.587173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:09.587173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:09.587173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:09.587173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:09.587173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:09.588159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:09.588159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:09.588159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:09.588159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:09.588159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:09.588159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:09.588159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:09.588159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:09.588159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:09.588159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:09.588159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:09.588159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:09.588159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:09.588159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:09.588159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:09.588159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:09.588159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:09.589490 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:09.589490 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:09.589490 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:09.589490 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:09.589490 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:09.589490 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:09.589490 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:09.589490 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:09.591656 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 272/532] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:09.592707 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1bf49-e6d4-47d5-a882-b8688ee0ad20 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:09.641157 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e00e3-5683-487e-8506-a3331184da06 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:09.674671 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b0d2f-8742-4749-869a-0642b5a3b36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:09.675409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b0d2f-8742-4749-869a-0642b5a3b36c None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:09.675751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b0d2f-8742-4749-869a-0642b5a3b36c None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:09.676150 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b0d2f-8742-4749-869a-0642b5a3b36c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:09.683779 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2b0d2f-8742-4749-869a-0642b5a3b36c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:09.685036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2b0d2f-8742-4749-869a-0642b5a3b36c None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:09.692524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1bf49-e6d4-47d5-a882-b8688ee0ad20 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:09.699913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-9b2b0d2f-8742-4749-869a-0642b5a3b36c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:09.699913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:09.699913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:09.699913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:09.699913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:09.699913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:09.699913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:09.699913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:09.699913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:09.699913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:09.699913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:09.699913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:09.699913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:09.699913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:09.699913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:09.700772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:09.700772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:09.700772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:09.700772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:09.700772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:09.700772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:09.700772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:09.700772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:09.700772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:09.700772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:09.700772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:09.700772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:09.700772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:09.700772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:09.700772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:09.700772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:09.700772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:09.701722 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:09.701722 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:09.701722 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:09.701722 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:09.701722 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:09.701722 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:09.701722 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:09.701722 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:09.705851 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 261/533] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:09.706741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b0d2f-8742-4749-869a-0642b5a3b36c None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:09.730218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059b6d0-d39f-4e90-a4e1-e9bcba835033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:09.731311 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059b6d0-d39f-4e90-a4e1-e9bcba835033 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:09.731812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059b6d0-d39f-4e90-a4e1-e9bcba835033 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:09.732277 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059b6d0-d39f-4e90-a4e1-e9bcba835033 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:09.733261 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059b6d0-d39f-4e90-a4e1-e9bcba835033 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:09.733793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059b6d0-d39f-4e90-a4e1-e9bcba835033 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:09.735443 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059b6d0-d39f-4e90-a4e1-e9bcba835033 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:09.749827 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 273/534] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:09.750965 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059b6d0-d39f-4e90-a4e1-e9bcba835033 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:09.806810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b0d2f-8742-4749-869a-0642b5a3b36c None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:09.840524 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf732777-89f0-43d4-948d-052d5aa327fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:09.841220 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf732777-89f0-43d4-948d-052d5aa327fd None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:09.841547 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf732777-89f0-43d4-948d-052d5aa327fd None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:09.841819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf732777-89f0-43d4-948d-052d5aa327fd None admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:09.842615 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf732777-89f0-43d4-948d-052d5aa327fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:09.843006 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf732777-89f0-43d4-948d-052d5aa327fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:09.844347 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf732777-89f0-43d4-948d-052d5aa327fd None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:09.849928 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059b6d0-d39f-4e90-a4e1-e9bcba835033 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:09.862683 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 262/535] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:09.863631 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf732777-89f0-43d4-948d-052d5aa327fd None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:09.884026 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1ccaf-9c63-4c59-acfc-9949a83f4653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:09.885160 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1ccaf-9c63-4c59-acfc-9949a83f4653 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:09.885654 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1ccaf-9c63-4c59-acfc-9949a83f4653 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:09.886135 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1ccaf-9c63-4c59-acfc-9949a83f4653 None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:09.887123 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f1ccaf-9c63-4c59-acfc-9949a83f4653 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:09.887625 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f1ccaf-9c63-4c59-acfc-9949a83f4653 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:09.889307 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f1ccaf-9c63-4c59-acfc-9949a83f4653 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:09.901965 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 274/536] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:09.903050 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1ccaf-9c63-4c59-acfc-9949a83f4653 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:09.966914 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf732777-89f0-43d4-948d-052d5aa327fd None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:09.997849 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1ccaf-9c63-4c59-acfc-9949a83f4653 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:10.000114 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2ca64-55ef-439d-b29d-83164fab0918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:10.001221 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2ca64-55ef-439d-b29d-83164fab0918 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:10.001221 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2ca64-55ef-439d-b29d-83164fab0918 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:10.001513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2ca64-55ef-439d-b29d-83164fab0918 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:10.008231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2ca64-55ef-439d-b29d-83164fab0918 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:10.009812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2ca64-55ef-439d-b29d-83164fab0918 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:10.023594 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-79e2ca64-55ef-439d-b29d-83164fab0918 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:10.023594 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:10.023594 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:10.023594 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:10.023594 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:10.023594 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:10.023594 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:10.023594 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:10.023594 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:10.023594 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:10.023594 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:10.023594 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:10.023594 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:10.023594 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:10.023594 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:10.024442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:10.024442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:10.024442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:10.024442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:10.024442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:10.024442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:10.024442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:10.024442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:10.024442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:10.024442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:10.024442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:10.024442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:10.024442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:10.024442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:10.024442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:10.024442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:10.024442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:10.025511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:10.025511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:10.025511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:10.025511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:10.025511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:10.025511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:10.025511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:10.025511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:10.025511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 263/537] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:10.025979 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2ca64-55ef-439d-b29d-83164fab0918 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:10.035708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f120b4-b001-421c-a068-756f6d0f4ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:10.036633 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f120b4-b001-421c-a068-756f6d0f4ffc None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:10.037137 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f120b4-b001-421c-a068-756f6d0f4ffc None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:10.037625 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f120b4-b001-421c-a068-756f6d0f4ffc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:10.038645 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f120b4-b001-421c-a068-756f6d0f4ffc None admin] RBAC: Adding query filter params (role.id=f55c3728df78466989d0cd1b13c5ea28, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=0f0d3f4264a04093894d6bc5c5c45c7c) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:10.039147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f120b4-b001-421c-a068-756f6d0f4ffc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:10.040847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f120b4-b001-421c-a068-756f6d0f4ffc None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:10.047068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 275/538] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:34:09 2022] GET /identity/v3/role_assignments?user.id=0f0d3f4264a04093894d6bc5c5c45c7c&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=f55c3728df78466989d0cd1b13c5ea28 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:10.048048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f120b4-b001-421c-a068-756f6d0f4ffc None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:10.130257 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2ca64-55ef-439d-b29d-83164fab0918 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:10.148344 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f120b4-b001-421c-a068-756f6d0f4ffc None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:10.164399 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212a019-7c7f-4ab0-864d-f0e48300e1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:10.165095 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212a019-7c7f-4ab0-864d-f0e48300e1c7 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:10.165374 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212a019-7c7f-4ab0-864d-f0e48300e1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:10.165651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212a019-7c7f-4ab0-864d-f0e48300e1c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:10.166559 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d212a019-7c7f-4ab0-864d-f0e48300e1c7 None admin] RBAC: Adding query filter params (role.id=f55c3728df78466989d0cd1b13c5ea28, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=7fd398c547554629927d2a80b3a8dee6) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:10.166867 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d212a019-7c7f-4ab0-864d-f0e48300e1c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:10.168359 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d212a019-7c7f-4ab0-864d-f0e48300e1c7 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:10.174820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 264/539] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:34:10 2022] GET /identity/v3/role_assignments?user.id=7fd398c547554629927d2a80b3a8dee6&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=f55c3728df78466989d0cd1b13c5ea28 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:10.186608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090b969-089e-4c3e-80d5-860b44351f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:10.187673 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090b969-089e-4c3e-80d5-860b44351f39 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:10.188151 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090b969-089e-4c3e-80d5-860b44351f39 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:10.188655 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090b969-089e-4c3e-80d5-860b44351f39 None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:10.189751 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090b969-089e-4c3e-80d5-860b44351f39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:10.190276 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090b969-089e-4c3e-80d5-860b44351f39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:10.192018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090b969-089e-4c3e-80d5-860b44351f39 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:10.207101 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 276/540] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:10.213042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212a019-7c7f-4ab0-864d-f0e48300e1c7 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:10.335223 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212a019-7c7f-4ab0-864d-f0e48300e1c7 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:10.374269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8fa215-ff45-4c75-8a63-bafb059f341c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:10.375023 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fa215-ff45-4c75-8a63-bafb059f341c None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:10.375302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fa215-ff45-4c75-8a63-bafb059f341c None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:10.375572 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fa215-ff45-4c75-8a63-bafb059f341c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:10.376674 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8fa215-ff45-4c75-8a63-bafb059f341c None admin] RBAC: Adding query filter params (role.id=f55c3728df78466989d0cd1b13c5ea28, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=a44d689993bd4d30858a770012ee3e89) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:10.377081 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8fa215-ff45-4c75-8a63-bafb059f341c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:10.378777 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8fa215-ff45-4c75-8a63-bafb059f341c None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:10.385634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 265/541] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:34:10 2022] GET /identity/v3/role_assignments?user.id=a44d689993bd4d30858a770012ee3e89&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=f55c3728df78466989d0cd1b13c5ea28 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:10.905377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8800b9-0ed0-4e2f-b390-ec06f5f7d2fe None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:10.905910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8800b9-0ed0-4e2f-b390-ec06f5f7d2fe None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:10.905910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8800b9-0ed0-4e2f-b390-ec06f5f7d2fe None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:10.907532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 277/542] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:10.913125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8492c-3e2b-4b1a-9afa-76a2afc87575 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:10.913412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8492c-3e2b-4b1a-9afa-76a2afc87575 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:10.913693 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8492c-3e2b-4b1a-9afa-76a2afc87575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:10.941694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-b1d8492c-3e2b-4b1a-9afa-76a2afc87575 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:10.942906 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-b1d8492c-3e2b-4b1a-9afa-76a2afc87575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:11.030251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 266/543] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:34:11.065568 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8800b9-0ed0-4e2f-b390-ec06f5f7d2fe None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:11.193979 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8800b9-0ed0-4e2f-b390-ec06f5f7d2fe None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:11.229938 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a7994-7192-4050-8f76-15d503fa5c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:11.230764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a7994-7192-4050-8f76-15d503fa5c88 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:11.231038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a7994-7192-4050-8f76-15d503fa5c88 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:11.231302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a7994-7192-4050-8f76-15d503fa5c88 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:11.243537 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a7994-7192-4050-8f76-15d503fa5c88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:11.251818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-600a7994-7192-4050-8f76-15d503fa5c88 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:11.251818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:11.251818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:11.251818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:11.251818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:11.251818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:11.251818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:11.251818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:11.251818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:11.251818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:11.251818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:11.251818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:11.251818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:11.251818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:11.251818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:11.251818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:11.252822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:11.252822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:11.252822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:11.252822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:11.252822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:11.252822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:11.252822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:11.252822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:11.252822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:11.252822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:11.252822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:11.252822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:11.252822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:11.252822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:11.252822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:11.252822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:11.254572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:11.254572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:11.254572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:11.254572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:11.254572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:11.254572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:11.254572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:11.254572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:11.254572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:11.254572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:11.254572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:11.254572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:11.254572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 278/544] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:34:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:11.259341 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8492c-3e2b-4b1a-9afa-76a2afc87575 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:11.367613 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8492c-3e2b-4b1a-9afa-76a2afc87575 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:11.411707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca4dac-b08e-41bf-8a4c-bff815f17cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:11.412386 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca4dac-b08e-41bf-8a4c-bff815f17cb4 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:11.412682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca4dac-b08e-41bf-8a4c-bff815f17cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:11.412954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca4dac-b08e-41bf-8a4c-bff815f17cb4 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:11.413710 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca4dac-b08e-41bf-8a4c-bff815f17cb4 None admin] RBAC: Adding query filter params (name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:11.414010 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca4dac-b08e-41bf-8a4c-bff815f17cb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:11.415377 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca4dac-b08e-41bf-8a4c-bff815f17cb4 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:11.427792 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 267/545] 10.222.0.26 () {56 vars in 1103 bytes} [Mon May 9 09:34:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:11.433267 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a7994-7192-4050-8f76-15d503fa5c88 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:11.461753 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d1716-25e5-4336-847a-9d35cbd6c053 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:11.461753 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d1716-25e5-4336-847a-9d35cbd6c053 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:11.461753 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d1716-25e5-4336-847a-9d35cbd6c053 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:11.463712 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 268/546] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:34:11.469367 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8f5de-ee7c-4b75-9c6a-ff320ae32c92 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:11.469762 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8f5de-ee7c-4b75-9c6a-ff320ae32c92 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:11.470128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8f5de-ee7c-4b75-9c6a-ff320ae32c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:11.489423 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-7de8f5de-ee7c-4b75-9c6a-ff320ae32c92 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:11.494775 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-7de8f5de-ee7c-4b75-9c6a-ff320ae32c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:11.574307 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a7994-7192-4050-8f76-15d503fa5c88 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:11.606610 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 269/547] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:11.608058 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601b872-a2a0-485a-ad61-b7d9a81b7760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:11.608892 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601b872-a2a0-485a-ad61-b7d9a81b7760 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:11.609170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601b872-a2a0-485a-ad61-b7d9a81b7760 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:11.609440 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601b872-a2a0-485a-ad61-b7d9a81b7760 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:11.619513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601b872-a2a0-485a-ad61-b7d9a81b7760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:11.621297 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601b872-a2a0-485a-ad61-b7d9a81b7760 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:11.639301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-5601b872-a2a0-485a-ad61-b7d9a81b7760 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:11.639301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:11.639301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:11.639301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:11.639301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:11.639301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:11.639301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:11.639301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:11.639301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:11.639301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:11.639301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:11.639301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:11.639301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:11.639301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:11.639301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:11.640542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:11.640542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:11.640542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:11.640542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:11.640542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:11.640542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:11.640542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:11.640542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:11.640542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:11.640542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:11.640542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:11.640542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:11.640542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:11.640542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:11.640542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:11.640542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:11.640542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:11.641730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:11.641730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:11.641730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:11.641730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:11.641730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 279/548] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:11.641991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8f5de-ee7c-4b75-9c6a-ff320ae32c92 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:11.646773 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601b872-a2a0-485a-ad61-b7d9a81b7760 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:11.751341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601b872-a2a0-485a-ad61-b7d9a81b7760 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:11.777883 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8f5de-ee7c-4b75-9c6a-ff320ae32c92 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:11.789701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a5377-6577-46b0-be68-dc3d441ef715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:11.790367 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a5377-6577-46b0-be68-dc3d441ef715 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:11.790644 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a5377-6577-46b0-be68-dc3d441ef715 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:11.790945 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a5377-6577-46b0-be68-dc3d441ef715 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:11.791669 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8a5377-6577-46b0-be68-dc3d441ef715 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:11.791966 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8a5377-6577-46b0-be68-dc3d441ef715 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:11.793371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8a5377-6577-46b0-be68-dc3d441ef715 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:11.809109 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 280/549] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:11.810585 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a5377-6577-46b0-be68-dc3d441ef715 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:11.821949 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89297df-2b73-41e2-b1fb-2154733a292b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:11.822811 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89297df-2b73-41e2-b1fb-2154733a292b None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:11.823098 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89297df-2b73-41e2-b1fb-2154733a292b None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:11.823357 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89297df-2b73-41e2-b1fb-2154733a292b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:11.857625 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89297df-2b73-41e2-b1fb-2154733a292b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:11.873601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-a89297df-2b73-41e2-b1fb-2154733a292b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:11.873601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:11.873601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:11.873601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:11.873601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:11.873601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:11.873601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:11.873601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:11.873601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:11.873601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:11.873601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:11.873601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:11.873601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:11.873601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:11.873601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:11.873601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:11.875751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:11.875751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:11.875751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:11.875751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:11.875751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:11.875751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:11.875751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:11.875751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:11.875751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:11.875751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:11.875751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:11.875751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:11.875751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:11.875751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:11.875751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:11.875751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:11.877765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:11.877765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:11.877765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:11.877765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:11.877765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:11.877765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:11.877765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:11.877765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:11.877765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:11.877765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:11.877765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:11.877765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:11.877765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 270/550] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:34:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:11.884802 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89297df-2b73-41e2-b1fb-2154733a292b None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:11.908923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a5377-6577-46b0-be68-dc3d441ef715 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:11.943929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b81ddf-4155-4e69-bcc6-4c7d8c1615f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:11.944758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b81ddf-4155-4e69-bcc6-4c7d8c1615f5 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:11.945039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b81ddf-4155-4e69-bcc6-4c7d8c1615f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:11.945310 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b81ddf-4155-4e69-bcc6-4c7d8c1615f5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:11.953119 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b81ddf-4155-4e69-bcc6-4c7d8c1615f5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:11.954613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b81ddf-4155-4e69-bcc6-4c7d8c1615f5 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:11.970636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-d7b81ddf-4155-4e69-bcc6-4c7d8c1615f5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 09:34:11.970636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:11.970636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:11.970636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:11.970636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:11.970636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:11.970636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:11.970636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:11.970636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:11.970636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:11.970636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:11.970636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:11.970636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:11.970636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:11.970636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:11.970636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:11.972857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:11.972857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:11.972857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:11.972857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:11.972857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:11.972857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:11.972857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:11.972857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:11.972857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:11.972857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:11.972857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:11.972857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:11.972857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:11.972857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:11.972857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:11.972857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:11.972857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:11.972857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:11.975083 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:11.975083 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:11.975083 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:11.975083 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:11.975083 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:11.975083 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:11.975083 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:11.975083 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:11.975083 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 09:34:11.975083 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:11.975083 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 281/551] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:34:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:11.982177 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b81ddf-4155-4e69-bcc6-4c7d8c1615f5 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:12.091820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b81ddf-4155-4e69-bcc6-4c7d8c1615f5 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:12.127092 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd46ceb0-da19-4532-87a2-49cc89cc66c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:12.127762 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46ceb0-da19-4532-87a2-49cc89cc66c6 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:12.128039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46ceb0-da19-4532-87a2-49cc89cc66c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:12.128755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46ceb0-da19-4532-87a2-49cc89cc66c6 None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:12.129607 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd46ceb0-da19-4532-87a2-49cc89cc66c6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:12.129912 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd46ceb0-da19-4532-87a2-49cc89cc66c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:12.131352 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd46ceb0-da19-4532-87a2-49cc89cc66c6 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:12.150431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 282/552] 10.222.0.26 () {56 vars in 1135 bytes} [Mon May 9 09:34:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:12.156091 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd46ceb0-da19-4532-87a2-49cc89cc66c6 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:12.174577 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89297df-2b73-41e2-b1fb-2154733a292b None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:12.240334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8697ca-dcae-462d-bd53-de1b34cddc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:12.241032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8697ca-dcae-462d-bd53-de1b34cddc53 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:12.241308 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8697ca-dcae-462d-bd53-de1b34cddc53 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:12.241576 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8697ca-dcae-462d-bd53-de1b34cddc53 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:12.242330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8697ca-dcae-462d-bd53-de1b34cddc53 None admin] RBAC: Adding query filter params (name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:12.242634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8697ca-dcae-462d-bd53-de1b34cddc53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:12.244355 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8697ca-dcae-462d-bd53-de1b34cddc53 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:12.253973 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 271/553] 10.222.0.26 () {56 vars in 1103 bytes} [Mon May 9 09:34:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:12.259056 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8697ca-dcae-462d-bd53-de1b34cddc53 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:12.289991 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd46ceb0-da19-4532-87a2-49cc89cc66c6 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:12.368138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be540ac-7ca0-4571-b82b-38b6a4f03083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:12.368747 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be540ac-7ca0-4571-b82b-38b6a4f03083 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:12.369068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be540ac-7ca0-4571-b82b-38b6a4f03083 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:12.369340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be540ac-7ca0-4571-b82b-38b6a4f03083 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:12.396720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8697ca-dcae-462d-bd53-de1b34cddc53 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:12.432630 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d454db0-0e7e-4eba-977f-de210734aa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:12.433457 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d454db0-0e7e-4eba-977f-de210734aa0a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:12.433732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d454db0-0e7e-4eba-977f-de210734aa0a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:12.433998 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d454db0-0e7e-4eba-977f-de210734aa0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:12.445571 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d454db0-0e7e-4eba-977f-de210734aa0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:12.447267 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d454db0-0e7e-4eba-977f-de210734aa0a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:12.466427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-6d454db0-0e7e-4eba-977f-de210734aa0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:12.466427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:12.466427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:12.466427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:12.466427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:12.466427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:12.466427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:12.466427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:12.466427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:12.466427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:12.466427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:12.466427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:12.466427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:12.466427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:12.466427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:12.467330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:12.467330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:12.467330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:12.467330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:12.467330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:12.467330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:12.467330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:12.467330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:12.467330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:12.467330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:12.467330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:12.467330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:12.467330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:12.467330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:12.467330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:12.467330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:12.467330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:12.468860 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:12.468860 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:12.468860 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:12.468860 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:12.468860 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 272/554] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:12.476334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d454db0-0e7e-4eba-977f-de210734aa0a None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:12.565599 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be540ac-7ca0-4571-b82b-38b6a4f03083 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:12.567086 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be540ac-7ca0-4571-b82b-38b6a4f03083 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:12.582999 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d454db0-0e7e-4eba-977f-de210734aa0a None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:12.619263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37954258-9d36-409e-8fd0-d3ae9fd29705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:12.619854 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37954258-9d36-409e-8fd0-d3ae9fd29705 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:12.620132 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37954258-9d36-409e-8fd0-d3ae9fd29705 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:12.621146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37954258-9d36-409e-8fd0-d3ae9fd29705 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:12.621879 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37954258-9d36-409e-8fd0-d3ae9fd29705 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:12.622184 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37954258-9d36-409e-8fd0-d3ae9fd29705 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:12.623625 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37954258-9d36-409e-8fd0-d3ae9fd29705 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:12.637011 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 273/555] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:12.642297 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37954258-9d36-409e-8fd0-d3ae9fd29705 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:12.755703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37954258-9d36-409e-8fd0-d3ae9fd29705 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:12.791781 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0656e-1c24-4dda-8eae-513b51a3aabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:12.793026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0656e-1c24-4dda-8eae-513b51a3aabd None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:12.793306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0656e-1c24-4dda-8eae-513b51a3aabd None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:12.793579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0656e-1c24-4dda-8eae-513b51a3aabd None admin] PATH_INFO: `/v3/users/nova` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:12.801270 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f0656e-1c24-4dda-8eae-513b51a3aabd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:12.802715 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f0656e-1c24-4dda-8eae-513b51a3aabd None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:12.818791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-57f0656e-1c24-4dda-8eae-513b51a3aabd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 09:34:12.818791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:12.818791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:12.818791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:12.818791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:12.818791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:12.818791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:12.818791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:12.818791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:12.818791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:12.818791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:12.818791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:12.818791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:12.818791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:12.818791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:12.818791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:12.819763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:12.819763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:12.819763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:12.819763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:12.819763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:12.819763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:12.819763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:12.819763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:12.819763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:12.819763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:12.819763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:12.819763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:12.819763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:12.819763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:12.819763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:12.819763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:12.819763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:12.819763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:12.820773 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:12.820773 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:12.820773 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:12.820773 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:12.820773 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:12.820773 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:12.820773 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:12.820773 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:12.820773 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 09:34:12.820773 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:12.820773 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 274/556] 10.222.0.26 () {56 vars in 1093 bytes} [Mon May 9 09:34:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:12.825473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 283/557] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:34:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 670 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:34:12.829635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0656e-1c24-4dda-8eae-513b51a3aabd None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:12.830803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be540ac-7ca0-4571-b82b-38b6a4f03083 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:12.924586 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be540ac-7ca0-4571-b82b-38b6a4f03083 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:12.926757 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0656e-1c24-4dda-8eae-513b51a3aabd None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:12.960799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832b82d-190d-468a-ba5f-d23d60a6df47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:12.961601 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832b82d-190d-468a-ba5f-d23d60a6df47 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:12.961880 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832b82d-190d-468a-ba5f-d23d60a6df47 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:12.962145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832b82d-190d-468a-ba5f-d23d60a6df47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:12.963348 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8245b42-5195-4eba-8034-c356bdbe1744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:12.964177 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8245b42-5195-4eba-8034-c356bdbe1744 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:12.964610 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8245b42-5195-4eba-8034-c356bdbe1744 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:12.965046 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8245b42-5195-4eba-8034-c356bdbe1744 None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:12.966126 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8245b42-5195-4eba-8034-c356bdbe1744 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:12.966583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8245b42-5195-4eba-8034-c356bdbe1744 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:12.968264 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8245b42-5195-4eba-8034-c356bdbe1744 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:12.973213 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832b82d-190d-468a-ba5f-d23d60a6df47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:12.974705 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832b82d-190d-468a-ba5f-d23d60a6df47 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:12.989174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 275/558] 10.222.0.26 () {56 vars in 1133 bytes} [Mon May 9 09:34:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:12.990799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5af991-7084-4ae7-b00d-67bc6e0bf84d None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:12.991248 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5af991-7084-4ae7-b00d-67bc6e0bf84d None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:12.991684 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5af991-7084-4ae7-b00d-67bc6e0bf84d None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:12.993480 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 276/559] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:12.994204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-8832b82d-190d-468a-ba5f-d23d60a6df47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:12.994204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:12.994204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:12.994204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:12.994204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:12.994204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:12.994204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:12.994204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:12.994204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:12.994204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:12.994204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:12.994204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:12.994204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:12.994204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:12.994204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:12.995051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:12.995051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:12.995051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:12.995051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:12.995051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:12.995051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:12.995051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:12.995051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:12.995051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:12.995051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:12.995051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:12.995051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:12.995051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:12.995051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:12.995051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:12.995051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:12.995051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:12.996211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:12.996211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:12.996211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:12.996211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:12.997353 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5af991-7084-4ae7-b00d-67bc6e0bf84d None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:13.000589 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 284/560] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:13.002401 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49014ae5-8c76-4391-94ac-066a593a0794 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:13.002401 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49014ae5-8c76-4391-94ac-066a593a0794 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:13.002582 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49014ae5-8c76-4391-94ac-066a593a0794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:13.019867 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-49014ae5-8c76-4391-94ac-066a593a0794 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:13.021219 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-49014ae5-8c76-4391-94ac-066a593a0794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:13.084975 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 285/561] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:13.085955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49014ae5-8c76-4391-94ac-066a593a0794 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:13.099819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5af991-7084-4ae7-b00d-67bc6e0bf84d None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:13.135974 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c54f22-5de3-4c85-b042-b46de05a1b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:13.137143 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c54f22-5de3-4c85-b042-b46de05a1b9a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:13.137521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c54f22-5de3-4c85-b042-b46de05a1b9a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:13.137792 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c54f22-5de3-4c85-b042-b46de05a1b9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:13.147610 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c54f22-5de3-4c85-b042-b46de05a1b9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:13.149104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c54f22-5de3-4c85-b042-b46de05a1b9a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:13.167698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-d9c54f22-5de3-4c85-b042-b46de05a1b9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:13.167698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:13.167698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:13.167698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:13.167698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:13.167698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:13.167698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:13.167698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:13.167698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:13.167698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:13.167698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:13.167698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:13.167698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:13.167698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:13.167698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:13.169269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:13.169269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:13.169269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:13.169269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:13.169269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:13.169269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:13.169269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:13.169269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:13.169269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:13.169269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:13.169269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:13.169269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:13.169269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:13.169269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:13.169269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:13.169269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:13.169269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:13.171242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:13.171242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:13.171242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:13.171242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:13.172828 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 277/562] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:13.174340 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c54f22-5de3-4c85-b042-b46de05a1b9a None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:13.189196 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49014ae5-8c76-4391-94ac-066a593a0794 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:13.221012 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e44451-283b-445b-9a5b-945bb40cc0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:13.221667 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e44451-283b-445b-9a5b-945bb40cc0e8 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:13.221942 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e44451-283b-445b-9a5b-945bb40cc0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:13.222212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e44451-283b-445b-9a5b-945bb40cc0e8 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:13.222935 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e44451-283b-445b-9a5b-945bb40cc0e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:13.223238 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e44451-283b-445b-9a5b-945bb40cc0e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:13.224631 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e44451-283b-445b-9a5b-945bb40cc0e8 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:13.235242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 286/563] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:13.236421 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76b350-1a67-4390-ae94-6df1cc6545fc None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:13.236728 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76b350-1a67-4390-ae94-6df1cc6545fc None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:13.237007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76b350-1a67-4390-ae94-6df1cc6545fc None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:13.238508 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 287/564] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:13.239174 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76b350-1a67-4390-ae94-6df1cc6545fc None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:13.271443 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c54f22-5de3-4c85-b042-b46de05a1b9a None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:13.303237 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8f0e9-c6d0-4dd9-9e01-f04fed58ed6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:13.304013 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8f0e9-c6d0-4dd9-9e01-f04fed58ed6b None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:13.304285 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8f0e9-c6d0-4dd9-9e01-f04fed58ed6b None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:13.304547 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8f0e9-c6d0-4dd9-9e01-f04fed58ed6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:13.314497 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8f0e9-c6d0-4dd9-9e01-f04fed58ed6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:13.322275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-5ba8f0e9-c6d0-4dd9-9e01-f04fed58ed6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:13.322275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:13.322275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:13.322275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:13.322275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:13.322275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:13.322275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:13.322275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:13.322275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:13.322275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:13.322275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:13.322275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:13.322275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:13.322275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:13.322275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:13.322275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:13.323183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:13.323183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:13.323183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:13.323183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:13.323183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:13.323183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:13.323183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:13.323183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:13.323183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:13.323183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:13.323183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:13.323183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:13.323183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:13.323183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:13.323183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:13.323183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:13.323996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:13.323996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:13.323996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:13.323996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:13.323996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:13.323996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:13.323996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:13.323996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:13.323996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:13.323996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:13.323996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:13.323996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:13.323996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 278/565] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:34:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:13.324893 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8f25e-204f-48a4-9f65-9dccd2201114 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:13.324963 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8f25e-204f-48a4-9f65-9dccd2201114 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:13.325246 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8f25e-204f-48a4-9f65-9dccd2201114 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:13.326417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 279/566] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:13.327094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8f25e-204f-48a4-9f65-9dccd2201114 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:13.338013 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76b350-1a67-4390-ae94-6df1cc6545fc None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:13.371813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa689b6-edb1-4fb1-a759-2e2862b8c738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:13.372465 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa689b6-edb1-4fb1-a759-2e2862b8c738 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:13.372764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa689b6-edb1-4fb1-a759-2e2862b8c738 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:13.373036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa689b6-edb1-4fb1-a759-2e2862b8c738 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:13.373786 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa689b6-edb1-4fb1-a759-2e2862b8c738 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:13.374085 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa689b6-edb1-4fb1-a759-2e2862b8c738 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:13.375457 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa689b6-edb1-4fb1-a759-2e2862b8c738 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:13.386489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 288/567] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:13.387801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb8433-3513-48b3-b1d7-eeca3ebf5694 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:13.388097 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb8433-3513-48b3-b1d7-eeca3ebf5694 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:13.388368 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb8433-3513-48b3-b1d7-eeca3ebf5694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:13.404743 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-2acb8433-3513-48b3-b1d7-eeca3ebf5694 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:13.405933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-2acb8433-3513-48b3-b1d7-eeca3ebf5694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:13.425225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8f25e-204f-48a4-9f65-9dccd2201114 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:13.456956 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d38963-e316-4ba6-8503-b0d2255843ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:13.457668 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d38963-e316-4ba6-8503-b0d2255843ee None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:13.457946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d38963-e316-4ba6-8503-b0d2255843ee None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:13.458213 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d38963-e316-4ba6-8503-b0d2255843ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:13.464705 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d38963-e316-4ba6-8503-b0d2255843ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:13.466158 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d38963-e316-4ba6-8503-b0d2255843ee None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:13.469391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 289/568] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:13.470203 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb8433-3513-48b3-b1d7-eeca3ebf5694 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:13.489360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-84d38963-e316-4ba6-8503-b0d2255843ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:13.489360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:13.489360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:13.489360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:13.489360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:13.489360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:13.489360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:13.489360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:13.489360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:13.489360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:13.489360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:13.489360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:13.489360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:13.489360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:13.489360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:13.490354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:13.490354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:13.490354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:13.490354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:13.490354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:13.490354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:13.490354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:13.490354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:13.490354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:13.490354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:13.490354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:13.490354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:13.490354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:13.490354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:13.490354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:13.490354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:13.490354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:13.491240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:13.491240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:13.491240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:13.491240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:13.491240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:13.491240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:13.491240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:13.491240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:13.491240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 280/569] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:13.492236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537dc9ae-0a70-4bee-a018-66c10cf3e722 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:13.492396 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537dc9ae-0a70-4bee-a018-66c10cf3e722 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:13.492694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537dc9ae-0a70-4bee-a018-66c10cf3e722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:13.510033 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-537dc9ae-0a70-4bee-a018-66c10cf3e722 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:13.511219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-537dc9ae-0a70-4bee-a018-66c10cf3e722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:13.571290 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb8433-3513-48b3-b1d7-eeca3ebf5694 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:13.573702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 281/570] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:13.574539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537dc9ae-0a70-4bee-a018-66c10cf3e722 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:13.603579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079e325-6dda-4f01-a271-37f03c711534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:13.604382 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079e325-6dda-4f01-a271-37f03c711534 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:13.604751 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079e325-6dda-4f01-a271-37f03c711534 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:13.605030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079e325-6dda-4f01-a271-37f03c711534 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:13.605778 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079e325-6dda-4f01-a271-37f03c711534 None admin] RBAC: Adding query filter params (name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:13.606081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079e325-6dda-4f01-a271-37f03c711534 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:13.607446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079e325-6dda-4f01-a271-37f03c711534 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:13.615144 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 290/571] 10.222.0.26 () {56 vars in 1103 bytes} [Mon May 9 09:34:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:13.615776 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079e325-6dda-4f01-a271-37f03c711534 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:13.677192 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537dc9ae-0a70-4bee-a018-66c10cf3e722 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:13.705801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079e325-6dda-4f01-a271-37f03c711534 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:13.712894 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8df01-5308-4979-b664-7882c73e02d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:13.713620 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8df01-5308-4979-b664-7882c73e02d1 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:13.713896 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8df01-5308-4979-b664-7882c73e02d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:13.714158 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8df01-5308-4979-b664-7882c73e02d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:13.720806 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8df01-5308-4979-b664-7882c73e02d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:13.722240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8df01-5308-4979-b664-7882c73e02d1 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:13.735666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-a7d8df01-5308-4979-b664-7882c73e02d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:13.735666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:13.735666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:13.735666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:13.735666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:13.735666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:13.735666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:13.735666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:13.735666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:13.735666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:13.735666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:13.735666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:13.735666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:13.735666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:13.735666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:13.736839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:13.736839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:13.736839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:13.736839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:13.736839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:13.736839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:13.736839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:13.736839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:13.736839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:13.736839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:13.736839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:13.736839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:13.736839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:13.736839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:13.736839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:13.736839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:13.736839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:13.738611 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:13.738611 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:13.738611 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:13.738611 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:13.738611 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:13.738611 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:13.738611 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:13.738611 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:13.738611 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 282/572] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:13.739104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5077fe8-4fd6-409e-a6cd-29c3bd554c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:13.739193 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8df01-5308-4979-b664-7882c73e02d1 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:13.739273 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5077fe8-4fd6-409e-a6cd-29c3bd554c2a None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:13.739273 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5077fe8-4fd6-409e-a6cd-29c3bd554c2a None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:13.739273 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5077fe8-4fd6-409e-a6cd-29c3bd554c2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:13.750579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5077fe8-4fd6-409e-a6cd-29c3bd554c2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:13.758293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-b5077fe8-4fd6-409e-a6cd-29c3bd554c2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:13.758293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:13.758293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:13.758293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:13.758293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:13.758293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:13.758293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:13.758293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:13.758293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:13.758293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:13.758293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:13.758293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:13.758293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:13.758293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:13.758293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:13.758293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:13.759391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:13.759391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:13.759391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:13.759391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:13.759391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:13.759391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:13.759391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:13.759391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:13.759391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:13.759391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:13.759391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:13.759391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:13.759391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:13.759391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:13.759391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:13.759391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:13.760356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:13.760356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:13.760356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:13.760356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:13.760356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:13.760356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:13.760356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:13.760356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:13.760356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:13.760356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:13.760356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:13.760356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:13.760356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 291/573] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:34:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:13.760356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5077fe8-4fd6-409e-a6cd-29c3bd554c2a None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:13.834732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8df01-5308-4979-b664-7882c73e02d1 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:13.857830 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5077fe8-4fd6-409e-a6cd-29c3bd554c2a None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:13.867901 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df4b37-3235-44ca-9422-8a1280749650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:13.868555 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df4b37-3235-44ca-9422-8a1280749650 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:13.868863 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df4b37-3235-44ca-9422-8a1280749650 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:13.869164 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df4b37-3235-44ca-9422-8a1280749650 None admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:13.869946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df4b37-3235-44ca-9422-8a1280749650 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:13.870253 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df4b37-3235-44ca-9422-8a1280749650 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:13.872044 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df4b37-3235-44ca-9422-8a1280749650 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:13.883474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 283/574] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:13.884225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df4b37-3235-44ca-9422-8a1280749650 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:13.891820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb628bd-6b42-458d-914a-ab9dfb2df578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:13.892618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb628bd-6b42-458d-914a-ab9dfb2df578 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:13.892925 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb628bd-6b42-458d-914a-ab9dfb2df578 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:13.893197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb628bd-6b42-458d-914a-ab9dfb2df578 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:13.902818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb628bd-6b42-458d-914a-ab9dfb2df578 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:13.910431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-edb628bd-6b42-458d-914a-ab9dfb2df578 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:13.910431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:13.910431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:13.910431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:13.910431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:13.910431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:13.910431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:13.910431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:13.910431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:13.910431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:13.910431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:13.910431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:13.910431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:13.910431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:13.910431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:13.910431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:13.911450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:13.911450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:13.911450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:13.911450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:13.911450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:13.911450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:13.911450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:13.911450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:13.911450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:13.911450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:13.911450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:13.911450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:13.911450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:13.911450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:13.911450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:13.911450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:13.912293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:13.912293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:13.912293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:13.912293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:13.912293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:13.912293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:13.912293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:13.912293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:13.912293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:13.912293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:13.912293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:13.912293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:13.912293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 292/575] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:34:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:13.914168 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb628bd-6b42-458d-914a-ab9dfb2df578 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:13.976572 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df4b37-3235-44ca-9422-8a1280749650 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:14.003294 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb628bd-6b42-458d-914a-ab9dfb2df578 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:14.008627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951589a0-9928-4225-b3ff-90e5c3cc087d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:14.009441 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951589a0-9928-4225-b3ff-90e5c3cc087d None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:14.009740 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951589a0-9928-4225-b3ff-90e5c3cc087d None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:14.010004 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951589a0-9928-4225-b3ff-90e5c3cc087d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:14.019564 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951589a0-9928-4225-b3ff-90e5c3cc087d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:14.021018 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951589a0-9928-4225-b3ff-90e5c3cc087d None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:14.034991 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3a5f5-3bb9-4a4e-8d1e-7a299d124dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:14.035629 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3a5f5-3bb9-4a4e-8d1e-7a299d124dd4 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:14.035910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3a5f5-3bb9-4a4e-8d1e-7a299d124dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:14.036180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3a5f5-3bb9-4a4e-8d1e-7a299d124dd4 None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:14.036991 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3a5f5-3bb9-4a4e-8d1e-7a299d124dd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:14.037292 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3a5f5-3bb9-4a4e-8d1e-7a299d124dd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:14.037727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-951589a0-9928-4225-b3ff-90e5c3cc087d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:14.037727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:14.037727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:14.037727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:14.037727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:14.037727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:14.037727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:14.037727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:14.037727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:14.037727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:14.037727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:14.037727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:14.037727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:14.037727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:14.037727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:14.038774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:14.038774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:14.038774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:14.038774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:14.038774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:14.038774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:14.038774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:14.038774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:14.038774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:14.038774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:14.038774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:14.038774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:14.038774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:14.038774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:14.038774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:14.038774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:14.038774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:14.039731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:14.039731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:14.039731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:14.039731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:14.039972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3a5f5-3bb9-4a4e-8d1e-7a299d124dd4 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:14.040043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 284/576] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:14.040043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951589a0-9928-4225-b3ff-90e5c3cc087d None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:14.054240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 293/577] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:14.055103 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3a5f5-3bb9-4a4e-8d1e-7a299d124dd4 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:14.131299 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951589a0-9928-4225-b3ff-90e5c3cc087d None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:14.146749 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3a5f5-3bb9-4a4e-8d1e-7a299d124dd4 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:14.164608 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f896e15-8e61-48ee-a659-4fef9b2b0380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:14.165278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f896e15-8e61-48ee-a659-4fef9b2b0380 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:14.165553 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f896e15-8e61-48ee-a659-4fef9b2b0380 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:14.165826 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f896e15-8e61-48ee-a659-4fef9b2b0380 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:14.166559 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f896e15-8e61-48ee-a659-4fef9b2b0380 None admin] RBAC: Adding query filter params (name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:14.166860 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f896e15-8e61-48ee-a659-4fef9b2b0380 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:14.168215 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f896e15-8e61-48ee-a659-4fef9b2b0380 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:14.175870 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 285/578] 10.222.0.26 () {56 vars in 1103 bytes} [Mon May 9 09:34:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:14.176614 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f896e15-8e61-48ee-a659-4fef9b2b0380 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:14.183460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7978af-f053-40bf-9ca0-0e8cfee61b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:14.184121 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7978af-f053-40bf-9ca0-0e8cfee61b65 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:14.184380 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7978af-f053-40bf-9ca0-0e8cfee61b65 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:14.184647 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7978af-f053-40bf-9ca0-0e8cfee61b65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:14.185498 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7978af-f053-40bf-9ca0-0e8cfee61b65 None admin] RBAC: Adding query filter params (role.id=f55c3728df78466989d0cd1b13c5ea28, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=380e04358f2349aa912d6a97518877cd) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:14.185804 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7978af-f053-40bf-9ca0-0e8cfee61b65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:14.187497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7978af-f053-40bf-9ca0-0e8cfee61b65 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:14.193014 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 294/579] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:34:14 2022] GET /identity/v3/role_assignments?user.id=380e04358f2349aa912d6a97518877cd&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=f55c3728df78466989d0cd1b13c5ea28 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:14.193846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7978af-f053-40bf-9ca0-0e8cfee61b65 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:14.287541 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f896e15-8e61-48ee-a659-4fef9b2b0380 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:14.302832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7978af-f053-40bf-9ca0-0e8cfee61b65 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:14.322075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f84ad-3049-4f82-8394-197c625e9d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:14.322762 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f84ad-3049-4f82-8394-197c625e9d46 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:14.323045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f84ad-3049-4f82-8394-197c625e9d46 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:14.323320 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f84ad-3049-4f82-8394-197c625e9d46 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:14.324078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f84ad-3049-4f82-8394-197c625e9d46 None admin] RBAC: Adding query filter params (name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:14.324382 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f84ad-3049-4f82-8394-197c625e9d46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:14.325841 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f84ad-3049-4f82-8394-197c625e9d46 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:14.334709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 286/580] 10.222.0.26 () {56 vars in 1103 bytes} [Mon May 9 09:34:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:14.335887 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f84ad-3049-4f82-8394-197c625e9d46 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:14.340011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec77ff2-2561-4c21-baef-81cadc3b3bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:14.340713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec77ff2-2561-4c21-baef-81cadc3b3bb4 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:14.340995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec77ff2-2561-4c21-baef-81cadc3b3bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:14.341267 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec77ff2-2561-4c21-baef-81cadc3b3bb4 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:14.342005 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec77ff2-2561-4c21-baef-81cadc3b3bb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:14.342309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec77ff2-2561-4c21-baef-81cadc3b3bb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:14.343718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec77ff2-2561-4c21-baef-81cadc3b3bb4 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:14.356367 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 295/581] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:14.357364 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec77ff2-2561-4c21-baef-81cadc3b3bb4 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:14.439167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f84ad-3049-4f82-8394-197c625e9d46 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:14.464100 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec77ff2-2561-4c21-baef-81cadc3b3bb4 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:14.470985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93ff59-5259-4ce4-85b1-6233a1908cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:14.471877 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93ff59-5259-4ce4-85b1-6233a1908cc1 None admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:14.472150 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93ff59-5259-4ce4-85b1-6233a1908cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:14.472417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93ff59-5259-4ce4-85b1-6233a1908cc1 None admin] PATH_INFO: `/v3/projects/9cd39464055141b8b43fde20d0885c99/users/6362303cb918486c8525225914332ad5/roles/f55c3728df78466989d0cd1b13c5ea28` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:14.487148 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f93ff59-5259-4ce4-85b1-6233a1908cc1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd39464055141b8b43fde20d0885c99, user_id=6362303cb918486c8525225914332ad5, role_id=f55c3728df78466989d0cd1b13c5ea28)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:14.488800 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f93ff59-5259-4ce4-85b1-6233a1908cc1 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:14.500641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45f872-2d74-4845-9b69-90f307af83b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:14.501514 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45f872-2d74-4845-9b69-90f307af83b0 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:14.501796 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45f872-2d74-4845-9b69-90f307af83b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:14.502118 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45f872-2d74-4845-9b69-90f307af83b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:14.513049 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 287/582] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:34:14 2022] PUT /identity/v3/projects/9cd39464055141b8b43fde20d0885c99/users/6362303cb918486c8525225914332ad5/roles/f55c3728df78466989d0cd1b13c5ea28 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:34:14.513049 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93ff59-5259-4ce4-85b1-6233a1908cc1 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:14.525589 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45f872-2d74-4845-9b69-90f307af83b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:14.528483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45f872-2d74-4845-9b69-90f307af83b0 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:14.552373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-bd45f872-2d74-4845-9b69-90f307af83b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:14.552373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:14.552373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:14.552373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:14.552373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:14.552373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:14.552373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:14.552373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:14.552373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:14.552373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:14.552373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:14.552373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:14.552373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:14.552373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:14.552373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:14.553731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:14.553731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:14.553731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:14.553731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:14.553731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:14.553731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:14.553731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:14.553731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:14.553731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:14.553731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:14.553731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:14.553731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:14.553731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:14.553731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:14.553731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:14.553731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:14.553731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:14.554950 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:14.554950 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:14.554950 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:14.554950 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:14.557201 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 296/583] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:14.559462 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45f872-2d74-4845-9b69-90f307af83b0 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:14.647385 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93ff59-5259-4ce4-85b1-6233a1908cc1 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:14.668432 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45f872-2d74-4845-9b69-90f307af83b0 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:14.685315 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e844ab9-66bc-4f3d-85a6-75666365dd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:14.686430 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e844ab9-66bc-4f3d-85a6-75666365dd16 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:14.686903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e844ab9-66bc-4f3d-85a6-75666365dd16 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:14.687306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e844ab9-66bc-4f3d-85a6-75666365dd16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:14.702818 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e844ab9-66bc-4f3d-85a6-75666365dd16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:14.704595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e844ab9-66bc-4f3d-85a6-75666365dd16 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:14.706514 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77629c2-f120-45b2-8a12-41159e5b52d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:14.708184 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77629c2-f120-45b2-8a12-41159e5b52d3 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:14.708715 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77629c2-f120-45b2-8a12-41159e5b52d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:14.709191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77629c2-f120-45b2-8a12-41159e5b52d3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:14.717988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77629c2-f120-45b2-8a12-41159e5b52d3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:14.719803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77629c2-f120-45b2-8a12-41159e5b52d3 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:14.724898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-7e844ab9-66bc-4f3d-85a6-75666365dd16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:14.724898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:14.724898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:14.724898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:14.724898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:14.724898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:14.724898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:14.724898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:14.724898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:14.724898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:14.724898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:14.724898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:14.724898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:14.724898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:14.724898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:14.725821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:14.725821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:14.725821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:14.725821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:14.725821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:14.725821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:14.725821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:14.725821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:14.725821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:14.725821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:14.725821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:14.725821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:14.725821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:14.725821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:14.725821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:14.725821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:14.725821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:14.726732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:14.726732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:14.726732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:14.726732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:14.726732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 288/584] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:14.727437 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e844ab9-66bc-4f3d-85a6-75666365dd16 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:14.738487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-a77629c2-f120-45b2-8a12-41159e5b52d3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 09:34:14.738487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:14.738487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:14.738487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:14.738487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:14.738487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:14.738487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:14.738487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:14.738487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:14.738487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:14.738487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:14.738487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:14.738487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:14.738487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:14.738487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:14.738487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:14.739569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:14.739569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:14.739569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:14.739569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:14.739569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:14.739569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:14.739569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:14.739569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:14.739569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:14.739569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:14.739569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:14.739569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:14.739569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:14.739569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:14.739569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:14.739569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:14.739569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:14.739569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:14.740530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:14.740530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:14.740530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:14.740530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:14.740530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:14.740530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:14.740530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:14.740530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:14.740530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 09:34:14.740530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:14.742050 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 297/585] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:34:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:14.744738 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77629c2-f120-45b2-8a12-41159e5b52d3 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:14.831597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e844ab9-66bc-4f3d-85a6-75666365dd16 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:14.864730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818b9ba-53c6-4cc3-90df-646ce91b1b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:14.865636 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818b9ba-53c6-4cc3-90df-646ce91b1b53 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:14.866258 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818b9ba-53c6-4cc3-90df-646ce91b1b53 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:14.866857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818b9ba-53c6-4cc3-90df-646ce91b1b53 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:14.868207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818b9ba-53c6-4cc3-90df-646ce91b1b53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:14.868941 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818b9ba-53c6-4cc3-90df-646ce91b1b53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:14.870461 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818b9ba-53c6-4cc3-90df-646ce91b1b53 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:14.872087 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77629c2-f120-45b2-8a12-41159e5b52d3 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:14.882736 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 289/586] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:14.883765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818b9ba-53c6-4cc3-90df-646ce91b1b53 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:14.913484 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517f656-0a04-4caf-ad16-607f996baee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:14.914443 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517f656-0a04-4caf-ad16-607f996baee6 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:14.914890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517f656-0a04-4caf-ad16-607f996baee6 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:14.915300 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517f656-0a04-4caf-ad16-607f996baee6 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:14.916199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517f656-0a04-4caf-ad16-607f996baee6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:14.916691 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517f656-0a04-4caf-ad16-607f996baee6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:14.918188 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517f656-0a04-4caf-ad16-607f996baee6 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:14.930476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 298/587] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:14.931675 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517f656-0a04-4caf-ad16-607f996baee6 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:15.004718 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818b9ba-53c6-4cc3-90df-646ce91b1b53 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:15.026626 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517f656-0a04-4caf-ad16-607f996baee6 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:15.046801 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01a9a3-55a7-4e7e-a21f-f54b042d36ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:15.048213 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01a9a3-55a7-4e7e-a21f-f54b042d36ef None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:15.048623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01a9a3-55a7-4e7e-a21f-f54b042d36ef None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:15.049143 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01a9a3-55a7-4e7e-a21f-f54b042d36ef None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:15.050241 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01a9a3-55a7-4e7e-a21f-f54b042d36ef None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:15.050793 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01a9a3-55a7-4e7e-a21f-f54b042d36ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:15.052933 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01a9a3-55a7-4e7e-a21f-f54b042d36ef None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:15.070333 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea32403-dd9a-4daf-8c7d-0781b7df56ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:15.071841 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea32403-dd9a-4daf-8c7d-0781b7df56ba None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:15.072280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea32403-dd9a-4daf-8c7d-0781b7df56ba None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:15.072650 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea32403-dd9a-4daf-8c7d-0781b7df56ba None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:15.079940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 290/588] 10.222.0.26 () {56 vars in 1137 bytes} [Mon May 9 09:34:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:15.079940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01a9a3-55a7-4e7e-a21f-f54b042d36ef None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:15.085149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea32403-dd9a-4daf-8c7d-0781b7df56ba None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:15.086639 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea32403-dd9a-4daf-8c7d-0781b7df56ba None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:15.104737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-3ea32403-dd9a-4daf-8c7d-0781b7df56ba None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 09:34:15.104737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:15.104737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:15.104737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:15.104737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:15.104737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:15.104737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:15.104737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:15.104737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:15.104737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:15.104737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:15.104737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:15.104737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:15.104737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:15.104737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:15.104737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:15.105926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:15.105926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:15.105926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:15.105926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:15.105926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:15.105926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:15.105926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:15.105926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:15.105926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:15.105926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:15.105926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:15.105926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:15.105926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:15.105926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:15.105926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:15.105926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:15.105926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:15.105926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:15.107130 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:15.107130 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:15.107130 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:15.107130 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:15.107130 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:15.107130 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:15.107130 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:15.107130 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:15.107130 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 09:34:15.107130 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:15.112985 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 299/589] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:34:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:15.113886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea32403-dd9a-4daf-8c7d-0781b7df56ba None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:15.191820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01a9a3-55a7-4e7e-a21f-f54b042d36ef None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:15.216150 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea32403-dd9a-4daf-8c7d-0781b7df56ba None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:15.225191 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28101046-95b6-4c9e-82d5-e9a7382df54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:15.226170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28101046-95b6-4c9e-82d5-e9a7382df54d None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:15.226594 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28101046-95b6-4c9e-82d5-e9a7382df54d None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:15.227003 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28101046-95b6-4c9e-82d5-e9a7382df54d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:15.234479 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28101046-95b6-4c9e-82d5-e9a7382df54d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:15.236142 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28101046-95b6-4c9e-82d5-e9a7382df54d None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:15.250758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07230d-0697-4222-b056-62ef6142c860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:15.251678 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07230d-0697-4222-b056-62ef6142c860 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:15.251902 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-28101046-95b6-4c9e-82d5-e9a7382df54d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 09:34:15.251902 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:15.251902 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:15.251902 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:15.251902 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:15.251902 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:15.251902 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:15.251902 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:15.251902 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:15.251902 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:15.251902 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:15.251902 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:15.251902 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:15.251902 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:15.251902 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:15.251902 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:15.253045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:15.253045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:15.253045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:15.253045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:15.253045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:15.253045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:15.253045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:15.253045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:15.253045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:15.253045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:15.253045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:15.253045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:15.253045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:15.253045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:15.253045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:15.253045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:15.253045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:15.253045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:15.254236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:15.254236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:15.254236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:15.254236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:15.254236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:15.254236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:15.254236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:15.254236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:15.254236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 09:34:15.254236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:15.254896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07230d-0697-4222-b056-62ef6142c860 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:15.254896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07230d-0697-4222-b056-62ef6142c860 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:15.256340 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 291/590] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:34:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:15.257450 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28101046-95b6-4c9e-82d5-e9a7382df54d None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:15.265693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07230d-0697-4222-b056-62ef6142c860 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:15.267348 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07230d-0697-4222-b056-62ef6142c860 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:15.285897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-4d07230d-0697-4222-b056-62ef6142c860 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:15.285897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:15.285897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:15.285897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:15.285897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:15.285897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:15.285897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:15.285897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:15.285897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:15.285897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:15.285897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:15.285897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:15.285897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:15.285897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:15.285897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:15.287018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:15.287018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:15.287018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:15.287018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:15.287018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:15.287018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:15.287018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:15.287018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:15.287018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:15.287018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:15.287018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:15.287018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:15.287018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:15.287018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:15.287018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:15.287018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:15.287018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:15.288148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:15.288148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:15.288148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:15.288148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:15.290143 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 300/591] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:15.290143 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07230d-0697-4222-b056-62ef6142c860 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:15.364588 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28101046-95b6-4c9e-82d5-e9a7382df54d None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:15.393062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07230d-0697-4222-b056-62ef6142c860 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:15.402402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4547c-a3b1-4c7c-bd72-b3cd0deb5c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:15.403295 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4547c-a3b1-4c7c-bd72-b3cd0deb5c60 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:15.403741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4547c-a3b1-4c7c-bd72-b3cd0deb5c60 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:15.404152 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4547c-a3b1-4c7c-bd72-b3cd0deb5c60 None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:15.405163 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4547c-a3b1-4c7c-bd72-b3cd0deb5c60 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:15.405602 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4547c-a3b1-4c7c-bd72-b3cd0deb5c60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:15.407184 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4547c-a3b1-4c7c-bd72-b3cd0deb5c60 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:15.425637 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 292/592] 10.222.0.26 () {56 vars in 1137 bytes} [Mon May 9 09:34:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:15.426747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4547c-a3b1-4c7c-bd72-b3cd0deb5c60 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:15.432283 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0e1ae-11cc-4e4b-a7eb-279d9be1fbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:15.433091 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0e1ae-11cc-4e4b-a7eb-279d9be1fbba None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:15.433440 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0e1ae-11cc-4e4b-a7eb-279d9be1fbba None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:15.433808 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0e1ae-11cc-4e4b-a7eb-279d9be1fbba None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:15.434732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0e1ae-11cc-4e4b-a7eb-279d9be1fbba None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:15.435112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0e1ae-11cc-4e4b-a7eb-279d9be1fbba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:15.436619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0e1ae-11cc-4e4b-a7eb-279d9be1fbba None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:15.454916 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 301/593] 10.222.0.26 () {56 vars in 1139 bytes} [Mon May 9 09:34:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:15.455796 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0e1ae-11cc-4e4b-a7eb-279d9be1fbba None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:15.525033 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4547c-a3b1-4c7c-bd72-b3cd0deb5c60 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:15.558466 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b5960-9f9a-4882-b9b9-bc6f69d8642c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:15.559373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b5960-9f9a-4882-b9b9-bc6f69d8642c None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:15.559800 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b5960-9f9a-4882-b9b9-bc6f69d8642c None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:15.560333 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b5960-9f9a-4882-b9b9-bc6f69d8642c None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:15.561237 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b5960-9f9a-4882-b9b9-bc6f69d8642c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:15.561820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b5960-9f9a-4882-b9b9-bc6f69d8642c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:15.562256 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0e1ae-11cc-4e4b-a7eb-279d9be1fbba None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:15.563511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b5960-9f9a-4882-b9b9-bc6f69d8642c None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:15.579865 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 293/594] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:15.581156 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b5960-9f9a-4882-b9b9-bc6f69d8642c None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:15.601369 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be7630-0b1f-4e4b-b782-aae653d71642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:15.602292 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be7630-0b1f-4e4b-b782-aae653d71642 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:15.602677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be7630-0b1f-4e4b-b782-aae653d71642 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:15.603022 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be7630-0b1f-4e4b-b782-aae653d71642 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:15.613495 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be7630-0b1f-4e4b-b782-aae653d71642 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:15.615120 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be7630-0b1f-4e4b-b782-aae653d71642 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:15.633501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-b0be7630-0b1f-4e4b-b782-aae653d71642 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:15.633501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:15.633501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:15.633501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:15.633501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:15.633501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:15.633501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:15.633501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:15.633501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:15.633501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:15.633501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:15.633501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:15.633501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:15.633501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:15.633501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:15.634693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:15.634693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:15.634693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:15.634693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:15.634693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:15.634693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:15.634693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:15.634693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:15.634693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:15.634693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:15.634693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:15.634693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:15.634693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:15.634693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:15.634693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:15.634693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:15.634693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:15.635839 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:15.635839 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:15.635839 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:15.635839 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:15.640991 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 302/595] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:15.641904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be7630-0b1f-4e4b-b782-aae653d71642 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:15.684139 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b5960-9f9a-4882-b9b9-bc6f69d8642c None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:15.723660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ba389-ca18-4c71-95b4-a56fd754964e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:15.724742 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ba389-ca18-4c71-95b4-a56fd754964e None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:15.725163 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ba389-ca18-4c71-95b4-a56fd754964e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:15.725574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ba389-ca18-4c71-95b4-a56fd754964e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:15.736395 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ba389-ca18-4c71-95b4-a56fd754964e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:15.738097 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ba389-ca18-4c71-95b4-a56fd754964e None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:15.755721 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be7630-0b1f-4e4b-b782-aae653d71642 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:15.756174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-9b8ba389-ca18-4c71-95b4-a56fd754964e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:15.756174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:15.756174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:15.756174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:15.756174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:15.756174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:15.756174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:15.756174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:15.756174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:15.756174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:15.756174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:15.756174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:15.756174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:15.756174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:15.756174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:15.757278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:15.757278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:15.757278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:15.757278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:15.757278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:15.757278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:15.757278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:15.757278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:15.757278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:15.757278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:15.757278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:15.757278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:15.757278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:15.757278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:15.757278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:15.757278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:15.757278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:15.758402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:15.758402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:15.758402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:15.758402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:15.760014 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 294/596] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:15.761014 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ba389-ca18-4c71-95b4-a56fd754964e None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:15.798501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ab8cc-2817-41c8-8aac-826ff0754765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:15.799398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ab8cc-2817-41c8-8aac-826ff0754765 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:15.799752 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ab8cc-2817-41c8-8aac-826ff0754765 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:15.800093 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ab8cc-2817-41c8-8aac-826ff0754765 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:15.809313 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ab8cc-2817-41c8-8aac-826ff0754765 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:15.811275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ab8cc-2817-41c8-8aac-826ff0754765 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:15.829073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-5b5ab8cc-2817-41c8-8aac-826ff0754765 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:15.829073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:15.829073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:15.829073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:15.829073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:15.829073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:15.829073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:15.829073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:15.829073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:15.829073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:15.829073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:15.829073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:15.829073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:15.829073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:15.829073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:15.830209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:15.830209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:15.830209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:15.830209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:15.830209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:15.830209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:15.830209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:15.830209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:15.830209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:15.830209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:15.830209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:15.830209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:15.830209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:15.830209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:15.830209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:15.830209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:15.830209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:15.831340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:15.831340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:15.831340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:15.831340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:15.831340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:15.831340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:15.831340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:15.831340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:15.836974 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 303/597] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:15.836974 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ab8cc-2817-41c8-8aac-826ff0754765 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:15.869200 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ba389-ca18-4c71-95b4-a56fd754964e None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:15.911254 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813ced6-3b9e-4ac9-a270-36b498a39f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:15.912361 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813ced6-3b9e-4ac9-a270-36b498a39f3d None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:15.913578 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813ced6-3b9e-4ac9-a270-36b498a39f3d None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:15.914003 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813ced6-3b9e-4ac9-a270-36b498a39f3d None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:15.914900 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813ced6-3b9e-4ac9-a270-36b498a39f3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:15.915347 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813ced6-3b9e-4ac9-a270-36b498a39f3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:15.916966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813ced6-3b9e-4ac9-a270-36b498a39f3d None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:15.933007 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 295/598] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:15.934129 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813ced6-3b9e-4ac9-a270-36b498a39f3d None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:15.946445 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ab8cc-2817-41c8-8aac-826ff0754765 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:15.979657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfeb68-9949-4501-b12e-75be2bddf143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:15.980419 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfeb68-9949-4501-b12e-75be2bddf143 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:15.981128 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfeb68-9949-4501-b12e-75be2bddf143 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:15.981128 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfeb68-9949-4501-b12e-75be2bddf143 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:15.982028 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfeb68-9949-4501-b12e-75be2bddf143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:15.982403 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfeb68-9949-4501-b12e-75be2bddf143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:15.983845 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfeb68-9949-4501-b12e-75be2bddf143 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:15.996892 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 304/599] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:15.997906 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfeb68-9949-4501-b12e-75be2bddf143 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:16.041093 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813ced6-3b9e-4ac9-a270-36b498a39f3d None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:16.075069 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa13dfd-40dd-45af-9110-fd65ee0f8d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:16.075978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa13dfd-40dd-45af-9110-fd65ee0f8d5b None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:16.076391 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa13dfd-40dd-45af-9110-fd65ee0f8d5b None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:16.076832 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa13dfd-40dd-45af-9110-fd65ee0f8d5b None admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:16.077782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa13dfd-40dd-45af-9110-fd65ee0f8d5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:16.078218 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa13dfd-40dd-45af-9110-fd65ee0f8d5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:16.079803 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa13dfd-40dd-45af-9110-fd65ee0f8d5b None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:16.096550 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfeb68-9949-4501-b12e-75be2bddf143 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:16.097313 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 296/600] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:16.098340 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa13dfd-40dd-45af-9110-fd65ee0f8d5b None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:16.130969 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b334a5-5263-4465-a853-be5d14cf979c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:16.131783 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b334a5-5263-4465-a853-be5d14cf979c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:16.132129 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b334a5-5263-4465-a853-be5d14cf979c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:16.132468 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b334a5-5263-4465-a853-be5d14cf979c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:16.140797 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b334a5-5263-4465-a853-be5d14cf979c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:16.143017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b334a5-5263-4465-a853-be5d14cf979c None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:16.161366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-48b334a5-5263-4465-a853-be5d14cf979c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:16.161366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:16.161366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:16.161366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:16.161366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:16.161366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:16.161366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:16.161366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:16.161366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:16.161366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:16.161366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:16.161366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:16.161366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:16.161366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:16.161366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:16.162517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:16.162517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:16.162517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:16.162517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:16.162517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:16.162517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:16.162517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:16.162517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:16.162517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:16.162517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:16.162517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:16.162517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:16.162517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:16.162517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:16.162517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:16.162517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:16.162517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:16.163594 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:16.163594 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:16.163594 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:16.163594 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:16.163594 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:16.163594 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:16.163594 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:16.163594 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:16.168837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 305/601] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:16.169743 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b334a5-5263-4465-a853-be5d14cf979c None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:16.205492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa13dfd-40dd-45af-9110-fd65ee0f8d5b None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:16.252725 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fc10a-4bcc-493c-8f6b-cf3c86439828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:16.253614 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fc10a-4bcc-493c-8f6b-cf3c86439828 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:16.253896 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fc10a-4bcc-493c-8f6b-cf3c86439828 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:16.254169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fc10a-4bcc-493c-8f6b-cf3c86439828 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:16.263940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fc10a-4bcc-493c-8f6b-cf3c86439828 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:16.266042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fc10a-4bcc-493c-8f6b-cf3c86439828 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:16.284438 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b334a5-5263-4465-a853-be5d14cf979c None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:16.285260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-9e5fc10a-4bcc-493c-8f6b-cf3c86439828 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:16.285260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:16.285260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:16.285260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:16.285260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:16.285260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:16.285260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:16.285260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:16.285260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:16.285260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:16.285260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:16.285260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:16.285260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:16.285260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:16.285260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:16.286225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:16.286225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:16.286225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:16.286225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:16.286225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:16.286225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:16.286225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:16.286225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:16.286225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:16.286225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:16.286225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:16.286225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:16.286225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:16.286225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:16.286225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:16.286225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:16.286225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:16.287290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:16.287290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:16.287290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:16.287290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:16.287290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:16.287290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:16.287290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:16.287290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:16.287290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 297/602] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:16.288021 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fc10a-4bcc-493c-8f6b-cf3c86439828 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:16.321559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3421a3-310b-4af9-a146-28939a5f978a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:16.322760 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3421a3-310b-4af9-a146-28939a5f978a None admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:16.323183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3421a3-310b-4af9-a146-28939a5f978a None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:16.323593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3421a3-310b-4af9-a146-28939a5f978a None admin] PATH_INFO: `/v3/projects/9cd39464055141b8b43fde20d0885c99/users/7fd398c547554629927d2a80b3a8dee6/roles/f55c3728df78466989d0cd1b13c5ea28` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:16.339382 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3421a3-310b-4af9-a146-28939a5f978a None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd39464055141b8b43fde20d0885c99, user_id=7fd398c547554629927d2a80b3a8dee6, role_id=f55c3728df78466989d0cd1b13c5ea28)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:16.341650 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3421a3-310b-4af9-a146-28939a5f978a None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:16.368698 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 306/603] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:34:16 2022] PUT /identity/v3/projects/9cd39464055141b8b43fde20d0885c99/users/7fd398c547554629927d2a80b3a8dee6/roles/f55c3728df78466989d0cd1b13c5ea28 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:34:16.368698 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3421a3-310b-4af9-a146-28939a5f978a None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:16.472441 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fc10a-4bcc-493c-8f6b-cf3c86439828 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:16.503513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3421a3-310b-4af9-a146-28939a5f978a None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:16.506995 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97516e74-dc1f-49c5-94e9-970f999d13eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:16.507661 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97516e74-dc1f-49c5-94e9-970f999d13eb None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:16.507933 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97516e74-dc1f-49c5-94e9-970f999d13eb None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:16.508200 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97516e74-dc1f-49c5-94e9-970f999d13eb None admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:16.509048 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97516e74-dc1f-49c5-94e9-970f999d13eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:16.509681 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97516e74-dc1f-49c5-94e9-970f999d13eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:16.511164 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97516e74-dc1f-49c5-94e9-970f999d13eb None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:16.523756 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 298/604] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:16.525269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59432ded-fc5d-49de-9f3e-833106d57c54 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:16.525967 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59432ded-fc5d-49de-9f3e-833106d57c54 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:16.525967 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59432ded-fc5d-49de-9f3e-833106d57c54 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:16.531549 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 299/605] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:16.532508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59432ded-fc5d-49de-9f3e-833106d57c54 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:16.556288 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79908eb7-f6ec-4ced-ad24-c916bafc6cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:16.557076 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79908eb7-f6ec-4ced-ad24-c916bafc6cc3 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:16.557357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79908eb7-f6ec-4ced-ad24-c916bafc6cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:16.557630 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79908eb7-f6ec-4ced-ad24-c916bafc6cc3 None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:16.558432 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79908eb7-f6ec-4ced-ad24-c916bafc6cc3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:16.558741 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79908eb7-f6ec-4ced-ad24-c916bafc6cc3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:16.560188 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79908eb7-f6ec-4ced-ad24-c916bafc6cc3 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:16.575062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 307/606] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:16.578036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea6d84-528a-438f-a52b-8fef1dd78fd0 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:16.578036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea6d84-528a-438f-a52b-8fef1dd78fd0 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:16.578036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea6d84-528a-438f-a52b-8fef1dd78fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:16.594043 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-37ea6d84-528a-438f-a52b-8fef1dd78fd0 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:16.595491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-37ea6d84-528a-438f-a52b-8fef1dd78fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:16.668856 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 308/607] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:16.670289 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea6d84-528a-438f-a52b-8fef1dd78fd0 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:16.685340 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59432ded-fc5d-49de-9f3e-833106d57c54 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:16.725275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5859b-6251-4eb6-b14c-88557ff8e927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:16.726315 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5859b-6251-4eb6-b14c-88557ff8e927 None admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:16.726593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5859b-6251-4eb6-b14c-88557ff8e927 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:16.726864 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5859b-6251-4eb6-b14c-88557ff8e927 None admin] PATH_INFO: `/v3/projects/9cd39464055141b8b43fde20d0885c99/users/a44d689993bd4d30858a770012ee3e89/roles/f55c3728df78466989d0cd1b13c5ea28` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:16.745268 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a5859b-6251-4eb6-b14c-88557ff8e927 None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd39464055141b8b43fde20d0885c99, user_id=a44d689993bd4d30858a770012ee3e89, role_id=f55c3728df78466989d0cd1b13c5ea28)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:16.747301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a5859b-6251-4eb6-b14c-88557ff8e927 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:16.764397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 300/608] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:34:16 2022] PUT /identity/v3/projects/9cd39464055141b8b43fde20d0885c99/users/a44d689993bd4d30858a770012ee3e89/roles/f55c3728df78466989d0cd1b13c5ea28 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:34:16.765639 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5859b-6251-4eb6-b14c-88557ff8e927 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:16.798174 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea6d84-528a-438f-a52b-8fef1dd78fd0 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:16.859544 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d8dec-9b4f-41ab-b1bc-050da309d7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:16.860447 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d8dec-9b4f-41ab-b1bc-050da309d7cf None admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:16.860761 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d8dec-9b4f-41ab-b1bc-050da309d7cf None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:16.861034 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d8dec-9b4f-41ab-b1bc-050da309d7cf None admin] PATH_INFO: `/v3/projects/9cd39464055141b8b43fde20d0885c99/users/0f0d3f4264a04093894d6bc5c5c45c7c/roles/f55c3728df78466989d0cd1b13c5ea28` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:16.877024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d8dec-9b4f-41ab-b1bc-050da309d7cf None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd39464055141b8b43fde20d0885c99, user_id=0f0d3f4264a04093894d6bc5c5c45c7c, role_id=f55c3728df78466989d0cd1b13c5ea28)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:16.878489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d8dec-9b4f-41ab-b1bc-050da309d7cf None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:16.896241 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 309/609] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:34:16 2022] PUT /identity/v3/projects/9cd39464055141b8b43fde20d0885c99/users/0f0d3f4264a04093894d6bc5c5c45c7c/roles/f55c3728df78466989d0cd1b13c5ea28 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:34:16.897983 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70798c48-ceca-4cd0-98e1-6a58d0971d61 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:16.898263 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70798c48-ceca-4cd0-98e1-6a58d0971d61 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:16.898533 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70798c48-ceca-4cd0-98e1-6a58d0971d61 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:16.904826 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 310/610] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:16.908776 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbecdb0-08c5-4989-8997-3d81fffa171b None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:16.909116 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbecdb0-08c5-4989-8997-3d81fffa171b None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:16.909329 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbecdb0-08c5-4989-8997-3d81fffa171b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:16.944572 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5859b-6251-4eb6-b14c-88557ff8e927 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:16.947521 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-8cbecdb0-08c5-4989-8997-3d81fffa171b None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:16.949796 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-8cbecdb0-08c5-4989-8997-3d81fffa171b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:17.008597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a596394-c6e0-46ff-917a-c21f26dd4c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:17.009358 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a596394-c6e0-46ff-917a-c21f26dd4c13 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:17.009681 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a596394-c6e0-46ff-917a-c21f26dd4c13 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:17.009949 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a596394-c6e0-46ff-917a-c21f26dd4c13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:17.023978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a596394-c6e0-46ff-917a-c21f26dd4c13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:17.034720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-0a596394-c6e0-46ff-917a-c21f26dd4c13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:17.034720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:17.034720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:17.034720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:17.034720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:17.034720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:17.034720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:17.034720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:17.034720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:17.034720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:17.034720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:17.034720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:17.034720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:17.034720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:17.034720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:17.034720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:17.035730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:17.035730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:17.035730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:17.035730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:17.035730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:17.035730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:17.035730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:17.035730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:17.035730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:17.035730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:17.035730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:17.035730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:17.035730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:17.035730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:17.035730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:17.035730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:17.036568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:17.036568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:17.036568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:17.036568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:17.036568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:17.036568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:17.036568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:17.036568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:17.036568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:17.036568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:17.036568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:17.036568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:17.041250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 301/611] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:34:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:17.043156 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a596394-c6e0-46ff-917a-c21f26dd4c13 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:17.045270 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 311/612] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:17.069498 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbecdb0-08c5-4989-8997-3d81fffa171b None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:17.172860 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a596394-c6e0-46ff-917a-c21f26dd4c13 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:17.207004 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbecdb0-08c5-4989-8997-3d81fffa171b None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:17.208036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483407c3-c15b-49d4-9a63-bf04c07f41e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:17.209007 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483407c3-c15b-49d4-9a63-bf04c07f41e2 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:17.209425 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483407c3-c15b-49d4-9a63-bf04c07f41e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:17.209830 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483407c3-c15b-49d4-9a63-bf04c07f41e2 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:17.210723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483407c3-c15b-49d4-9a63-bf04c07f41e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:17.211179 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483407c3-c15b-49d4-9a63-bf04c07f41e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:17.212793 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483407c3-c15b-49d4-9a63-bf04c07f41e2 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:17.221838 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 302/613] 10.222.0.26 () {56 vars in 1103 bytes} [Mon May 9 09:34:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:17.229618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483407c3-c15b-49d4-9a63-bf04c07f41e2 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:17.248869 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5f36b-a807-4a72-84e5-18af50efb146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:17.249980 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5f36b-a807-4a72-84e5-18af50efb146 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:17.250438 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5f36b-a807-4a72-84e5-18af50efb146 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:17.250849 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5f36b-a807-4a72-84e5-18af50efb146 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:17.261881 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5f36b-a807-4a72-84e5-18af50efb146 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:17.270042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-2aa5f36b-a807-4a72-84e5-18af50efb146 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:17.270042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:17.270042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:17.270042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:17.270042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:17.270042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:17.270042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:17.270042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:17.270042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:17.270042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:17.270042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:17.270042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:17.270042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:17.270042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:17.270042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:17.270042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:17.271211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:17.271211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:17.271211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:17.271211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:17.271211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:17.271211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:17.271211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:17.271211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:17.271211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:17.271211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:17.271211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:17.271211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:17.271211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:17.271211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:17.271211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:17.271211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:17.272204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:17.272204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:17.272204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:17.272204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:17.272204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:17.272204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:17.272204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:17.272204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:17.272204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:17.272204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:17.272204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:17.272204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:17.274473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 312/614] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:34:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:17.280269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5f36b-a807-4a72-84e5-18af50efb146 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:17.330692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483407c3-c15b-49d4-9a63-bf04c07f41e2 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:17.363033 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c7ae4-6886-4a17-b0d4-3349be08364f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:17.363829 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c7ae4-6886-4a17-b0d4-3349be08364f None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:17.364105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c7ae4-6886-4a17-b0d4-3349be08364f None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:17.364410 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c7ae4-6886-4a17-b0d4-3349be08364f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:17.375372 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c7ae4-6886-4a17-b0d4-3349be08364f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:17.376866 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c7ae4-6886-4a17-b0d4-3349be08364f None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:17.380691 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5f36b-a807-4a72-84e5-18af50efb146 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:17.393729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-067c7ae4-6886-4a17-b0d4-3349be08364f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:17.393729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:17.393729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:17.393729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:17.393729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:17.393729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:17.393729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:17.393729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:17.393729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:17.393729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:17.393729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:17.393729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:17.393729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:17.393729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:17.393729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:17.394857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:17.394857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:17.394857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:17.394857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:17.394857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:17.394857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:17.394857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:17.394857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:17.394857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:17.394857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:17.394857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:17.394857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:17.394857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:17.394857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:17.394857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:17.394857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:17.394857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:17.395971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:17.395971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:17.395971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:17.395971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:17.400059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 303/615] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:17.401307 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c7ae4-6886-4a17-b0d4-3349be08364f None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:17.419113 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dca79-9335-4edc-9cfe-ecd69974c6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:17.420070 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dca79-9335-4edc-9cfe-ecd69974c6c4 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:17.420534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dca79-9335-4edc-9cfe-ecd69974c6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:17.420990 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dca79-9335-4edc-9cfe-ecd69974c6c4 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:17.421927 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52dca79-9335-4edc-9cfe-ecd69974c6c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:17.422229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52dca79-9335-4edc-9cfe-ecd69974c6c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:17.423641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52dca79-9335-4edc-9cfe-ecd69974c6c4 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:17.434063 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 313/616] 10.222.0.26 () {56 vars in 1103 bytes} [Mon May 9 09:34:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:17.439211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dca79-9335-4edc-9cfe-ecd69974c6c4 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:17.538258 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dca79-9335-4edc-9cfe-ecd69974c6c4 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:17.566305 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c7ae4-6886-4a17-b0d4-3349be08364f None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:17.570831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09b35a-15e7-4cfd-8c90-da47668c84ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:17.571627 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09b35a-15e7-4cfd-8c90-da47668c84ef None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:17.571956 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09b35a-15e7-4cfd-8c90-da47668c84ef None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:17.572235 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09b35a-15e7-4cfd-8c90-da47668c84ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:17.582726 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09b35a-15e7-4cfd-8c90-da47668c84ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:17.584173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09b35a-15e7-4cfd-8c90-da47668c84ef None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:17.600692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-ed09b35a-15e7-4cfd-8c90-da47668c84ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:17.600692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:17.600692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:17.600692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:17.600692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:17.600692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:17.600692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:17.600692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:17.600692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:17.600692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:17.600692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:17.600692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:17.600692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:17.600692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:17.600692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:17.601613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:17.601613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:17.601613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:17.601613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:17.601613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:17.601613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:17.601613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:17.601613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:17.601613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:17.601613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:17.601613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:17.601613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:17.601613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:17.601613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:17.601613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:17.601613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:17.601613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:17.602508 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:17.602508 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:17.602508 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:17.602508 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:17.602508 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 314/617] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:17.607522 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09b35a-15e7-4cfd-8c90-da47668c84ef None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:17.633366 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460566d4-ce9f-44a3-b662-d2f9af6640d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:17.634460 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460566d4-ce9f-44a3-b662-d2f9af6640d0 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:17.634945 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460566d4-ce9f-44a3-b662-d2f9af6640d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:17.635416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460566d4-ce9f-44a3-b662-d2f9af6640d0 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:17.636341 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460566d4-ce9f-44a3-b662-d2f9af6640d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:17.636880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460566d4-ce9f-44a3-b662-d2f9af6640d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:17.638480 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460566d4-ce9f-44a3-b662-d2f9af6640d0 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:17.668764 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 304/618] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:17.673945 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460566d4-ce9f-44a3-b662-d2f9af6640d0 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:17.707127 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09b35a-15e7-4cfd-8c90-da47668c84ef None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:17.739577 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1856679-44f4-4866-b2c4-e71125fdf42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:17.740245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1856679-44f4-4866-b2c4-e71125fdf42b None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:17.740523 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1856679-44f4-4866-b2c4-e71125fdf42b None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:17.740828 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1856679-44f4-4866-b2c4-e71125fdf42b None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:17.741690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1856679-44f4-4866-b2c4-e71125fdf42b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:17.741993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1856679-44f4-4866-b2c4-e71125fdf42b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:17.743530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1856679-44f4-4866-b2c4-e71125fdf42b None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:17.764280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 315/619] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:17.770134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1856679-44f4-4866-b2c4-e71125fdf42b None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:17.874751 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1856679-44f4-4866-b2c4-e71125fdf42b None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:17.910744 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65563219-4366-4222-9190-6ea5942fc0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:17.911558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65563219-4366-4222-9190-6ea5942fc0d4 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:17.911835 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65563219-4366-4222-9190-6ea5942fc0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:17.912106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65563219-4366-4222-9190-6ea5942fc0d4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:17.920133 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65563219-4366-4222-9190-6ea5942fc0d4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:17.921913 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65563219-4366-4222-9190-6ea5942fc0d4 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:17.934739 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460566d4-ce9f-44a3-b662-d2f9af6640d0 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:17.940683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-65563219-4366-4222-9190-6ea5942fc0d4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 09:34:17.940683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:17.940683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:17.940683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:17.940683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:17.940683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:17.940683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:17.940683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:17.940683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:17.940683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:17.940683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:17.940683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:17.940683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:17.940683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:17.940683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:17.940683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:17.941661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:17.941661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:17.941661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:17.941661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:17.941661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:17.941661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:17.941661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:17.941661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:17.941661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:17.941661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:17.941661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:17.941661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:17.941661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:17.941661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:17.941661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:17.941661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:17.941661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:17.941661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:17.942674 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:17.942674 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:17.942674 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:17.942674 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:17.942674 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:17.942674 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:17.942674 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:17.942674 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:17.942674 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 09:34:17.942674 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:17.942674 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 316/620] 10.222.0.26 () {56 vars in 1093 bytes} [Mon May 9 09:34:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:17.943816 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65563219-4366-4222-9190-6ea5942fc0d4 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:17.991209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff20835-6bc1-48ec-bd23-4ff36e0d2b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:17.992463 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff20835-6bc1-48ec-bd23-4ff36e0d2b12 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:17.992971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff20835-6bc1-48ec-bd23-4ff36e0d2b12 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:17.993459 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff20835-6bc1-48ec-bd23-4ff36e0d2b12 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:18.008544 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff20835-6bc1-48ec-bd23-4ff36e0d2b12 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:18.014049 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff20835-6bc1-48ec-bd23-4ff36e0d2b12 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:18.039089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-9ff20835-6bc1-48ec-bd23-4ff36e0d2b12 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 09:34:18.039089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:18.039089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:18.039089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:18.039089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:18.039089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:18.039089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:18.039089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:18.039089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:18.039089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:18.039089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:18.039089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:18.039089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:18.039089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:18.039089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:18.039089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:18.040169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:18.040169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:18.040169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:18.040169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:18.040169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:18.040169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:18.040169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:18.040169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:18.040169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:18.040169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:18.040169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:18.040169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:18.040169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:18.040169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:18.040169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:18.040169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:18.040169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:18.040169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:18.041245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:18.041245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:18.041245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:18.041245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:18.041245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:18.041245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:18.041245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:18.041245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:18.041245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 09:34:18.041245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:18.051920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 305/621] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:34:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:18.057718 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff20835-6bc1-48ec-bd23-4ff36e0d2b12 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:18.163526 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff20835-6bc1-48ec-bd23-4ff36e0d2b12 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:18.174857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65563219-4366-4222-9190-6ea5942fc0d4 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:18.208283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e9c07-b5bd-44b4-9f26-3898c587ca15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:18.209385 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e9c07-b5bd-44b4-9f26-3898c587ca15 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:18.209882 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e9c07-b5bd-44b4-9f26-3898c587ca15 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:18.210390 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e9c07-b5bd-44b4-9f26-3898c587ca15 None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:18.211429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e9c07-b5bd-44b4-9f26-3898c587ca15 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:18.213027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e9c07-b5bd-44b4-9f26-3898c587ca15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:18.218152 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e9c07-b5bd-44b4-9f26-3898c587ca15 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:18.238259 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 306/622] 10.222.0.26 () {56 vars in 1135 bytes} [Mon May 9 09:34:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:18.243480 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e9c07-b5bd-44b4-9f26-3898c587ca15 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:18.259036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e6e79c-29f2-4d49-a54c-3129e5efdc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:18.261095 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6e79c-29f2-4d49-a54c-3129e5efdc7c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:18.261902 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6e79c-29f2-4d49-a54c-3129e5efdc7c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:18.262525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6e79c-29f2-4d49-a54c-3129e5efdc7c None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:18.263741 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e6e79c-29f2-4d49-a54c-3129e5efdc7c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:18.265779 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e6e79c-29f2-4d49-a54c-3129e5efdc7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:18.267458 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e6e79c-29f2-4d49-a54c-3129e5efdc7c None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:18.305607 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 317/623] 10.222.0.26 () {56 vars in 1133 bytes} [Mon May 9 09:34:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:18.311335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e6e79c-29f2-4d49-a54c-3129e5efdc7c None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:18.421542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e6e79c-29f2-4d49-a54c-3129e5efdc7c None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:18.464382 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2172e-9a3a-41cb-9493-1e697249ff2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:18.465456 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2172e-9a3a-41cb-9493-1e697249ff2b None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:18.465962 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2172e-9a3a-41cb-9493-1e697249ff2b None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:18.466442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2172e-9a3a-41cb-9493-1e697249ff2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:18.484184 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e9c07-b5bd-44b4-9f26-3898c587ca15 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:18.549457 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b2172e-9a3a-41cb-9493-1e697249ff2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:18.551363 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b2172e-9a3a-41cb-9493-1e697249ff2b None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:18.588568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8407b-a372-4788-99a2-adf368c62187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:18.589397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8407b-a372-4788-99a2-adf368c62187 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:18.589670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8407b-a372-4788-99a2-adf368c62187 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:18.589937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8407b-a372-4788-99a2-adf368c62187 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:18.621666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8407b-a372-4788-99a2-adf368c62187 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:18.623147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8407b-a372-4788-99a2-adf368c62187 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:18.660920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-9ef8407b-a372-4788-99a2-adf368c62187 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:18.660920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:18.660920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:18.660920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:18.660920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:18.660920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:18.660920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:18.660920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:18.660920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:18.660920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:18.660920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:18.660920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:18.660920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:18.660920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:18.660920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:18.661927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:18.661927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:18.661927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:18.661927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:18.661927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:18.661927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:18.661927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:18.661927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:18.661927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:18.661927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:18.661927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:18.661927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:18.661927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:18.661927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:18.661927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:18.661927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:18.661927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:18.662846 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:18.662846 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:18.662846 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:18.662846 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:18.662846 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 307/624] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:18.663819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 318/625] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:34:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:34:18.667971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8407b-a372-4788-99a2-adf368c62187 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:18.675023 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2172e-9a3a-41cb-9493-1e697249ff2b None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:18.836154 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8407b-a372-4788-99a2-adf368c62187 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:18.856337 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2172e-9a3a-41cb-9493-1e697249ff2b None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:18.887763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bb063-2a07-4ed2-85ff-7d8a0e16924b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:18.888565 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bb063-2a07-4ed2-85ff-7d8a0e16924b None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:18.888861 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bb063-2a07-4ed2-85ff-7d8a0e16924b None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:18.889129 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bb063-2a07-4ed2-85ff-7d8a0e16924b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:18.907139 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84bb063-2a07-4ed2-85ff-7d8a0e16924b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:18.908617 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84bb063-2a07-4ed2-85ff-7d8a0e16924b None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:18.918651 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d197b0-e86a-4c68-a900-4b846d7a1c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:18.919319 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d197b0-e86a-4c68-a900-4b846d7a1c81 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:18.919595 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d197b0-e86a-4c68-a900-4b846d7a1c81 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:18.919865 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d197b0-e86a-4c68-a900-4b846d7a1c81 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:18.920583 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d197b0-e86a-4c68-a900-4b846d7a1c81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:18.921017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d197b0-e86a-4c68-a900-4b846d7a1c81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:18.922382 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d197b0-e86a-4c68-a900-4b846d7a1c81 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:18.934276 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 319/626] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:18.935709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-d84bb063-2a07-4ed2-85ff-7d8a0e16924b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:18.935709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:18.935709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:18.935709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:18.935709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:18.935709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:18.935709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:18.935709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:18.935709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:18.935709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:18.935709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:18.935709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:18.935709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:18.935709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:18.935709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:18.936579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:18.936579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:18.936579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:18.936579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:18.936579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:18.936579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:18.936579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:18.936579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:18.936579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:18.936579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:18.936579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:18.936579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:18.936579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:18.936579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:18.936579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:18.936579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:18.936579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:18.937741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:18.937741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:18.937741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:18.937741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:18.937741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 308/627] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:18.941540 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d197b0-e86a-4c68-a900-4b846d7a1c81 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:18.942725 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bb063-2a07-4ed2-85ff-7d8a0e16924b None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:19.070443 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bb063-2a07-4ed2-85ff-7d8a0e16924b None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:19.082229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d197b0-e86a-4c68-a900-4b846d7a1c81 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:19.110243 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fcd26e-e1c3-4aa8-865e-839f12e1a451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:19.111002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcd26e-e1c3-4aa8-865e-839f12e1a451 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:19.111346 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcd26e-e1c3-4aa8-865e-839f12e1a451 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:19.111685 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcd26e-e1c3-4aa8-865e-839f12e1a451 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:19.112502 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fcd26e-e1c3-4aa8-865e-839f12e1a451 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:19.112901 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fcd26e-e1c3-4aa8-865e-839f12e1a451 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:19.114346 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fcd26e-e1c3-4aa8-865e-839f12e1a451 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:19.119221 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787228ab-11e3-4a68-a729-804152b3ee0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:19.120250 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787228ab-11e3-4a68-a729-804152b3ee0d None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:19.121139 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787228ab-11e3-4a68-a729-804152b3ee0d None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:19.122166 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787228ab-11e3-4a68-a729-804152b3ee0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:19.126913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 309/628] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:19.128391 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02254bf4-4c55-46c7-a59d-59d7e3e56bb2 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:19.128853 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02254bf4-4c55-46c7-a59d-59d7e3e56bb2 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:19.129193 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02254bf4-4c55-46c7-a59d-59d7e3e56bb2 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:19.134679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 310/629] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:19.134679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02254bf4-4c55-46c7-a59d-59d7e3e56bb2 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:19.134953 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787228ab-11e3-4a68-a729-804152b3ee0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:19.134953 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787228ab-11e3-4a68-a729-804152b3ee0d None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:19.151302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-787228ab-11e3-4a68-a729-804152b3ee0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:19.151302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:19.151302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:19.151302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:19.151302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:19.151302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:19.151302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:19.151302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:19.151302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:19.151302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:19.151302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:19.151302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:19.151302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:19.151302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:19.151302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:19.152293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:19.152293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:19.152293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:19.152293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:19.152293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:19.152293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:19.152293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:19.152293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:19.152293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:19.152293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:19.152293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:19.152293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:19.152293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:19.152293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:19.152293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:19.152293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:19.152293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:19.153371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:19.153371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:19.153371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:19.153371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:19.153371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:19.153371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:19.153371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:19.153371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:19.157542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 320/630] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:19.159138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13197341-68ff-49e4-80ce-42d4bf8d8b4c None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:19.159340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13197341-68ff-49e4-80ce-42d4bf8d8b4c None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:19.159605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13197341-68ff-49e4-80ce-42d4bf8d8b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:19.180269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-13197341-68ff-49e4-80ce-42d4bf8d8b4c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:19.181596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-13197341-68ff-49e4-80ce-42d4bf8d8b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:19.262349 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02254bf4-4c55-46c7-a59d-59d7e3e56bb2 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:19.262823 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 321/631] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:19.264322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13197341-68ff-49e4-80ce-42d4bf8d8b4c None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:19.293591 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47acb0-b840-43a8-8ed7-996e2f30434b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:19.294393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47acb0-b840-43a8-8ed7-996e2f30434b None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:19.294673 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47acb0-b840-43a8-8ed7-996e2f30434b None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:19.294944 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47acb0-b840-43a8-8ed7-996e2f30434b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:19.302817 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e47acb0-b840-43a8-8ed7-996e2f30434b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:19.304453 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e47acb0-b840-43a8-8ed7-996e2f30434b None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:19.323185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-7e47acb0-b840-43a8-8ed7-996e2f30434b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:19.323185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:19.323185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:19.323185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:19.323185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:19.323185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:19.323185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:19.323185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:19.323185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:19.323185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:19.323185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:19.323185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:19.323185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:19.323185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:19.323185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:19.324122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:19.324122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:19.324122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:19.324122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:19.324122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:19.324122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:19.324122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:19.324122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:19.324122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:19.324122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:19.324122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:19.324122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:19.324122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:19.324122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:19.324122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:19.324122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:19.324122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:19.325607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:19.325607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:19.325607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:19.325607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:19.325607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:19.325607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:19.325607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:19.325607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:19.325607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 311/632] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:19.325607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47acb0-b840-43a8-8ed7-996e2f30434b None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:19.380542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13197341-68ff-49e4-80ce-42d4bf8d8b4c None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:19.411591 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711c5e5-120c-4d81-8c4f-6bd50cc0c5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:19.412247 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711c5e5-120c-4d81-8c4f-6bd50cc0c5aa None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:19.412537 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711c5e5-120c-4d81-8c4f-6bd50cc0c5aa None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:19.412865 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711c5e5-120c-4d81-8c4f-6bd50cc0c5aa None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:19.415439 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711c5e5-120c-4d81-8c4f-6bd50cc0c5aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:19.415773 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711c5e5-120c-4d81-8c4f-6bd50cc0c5aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:19.417258 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711c5e5-120c-4d81-8c4f-6bd50cc0c5aa None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:19.422125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47acb0-b840-43a8-8ed7-996e2f30434b None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:19.432685 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 322/633] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:19.432685 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711c5e5-120c-4d81-8c4f-6bd50cc0c5aa None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:19.465458 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84407ec-2e15-4352-9e47-dfbc289b2b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:19.466272 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84407ec-2e15-4352-9e47-dfbc289b2b75 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:19.466550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84407ec-2e15-4352-9e47-dfbc289b2b75 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:19.466822 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84407ec-2e15-4352-9e47-dfbc289b2b75 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:19.478034 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84407ec-2e15-4352-9e47-dfbc289b2b75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:19.485804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-d84407ec-2e15-4352-9e47-dfbc289b2b75 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:19.485804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:19.485804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:19.485804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:19.485804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:19.485804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:19.485804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:19.485804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:19.485804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:19.485804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:19.485804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:19.485804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:19.485804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:19.485804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:19.485804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:19.485804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:19.486768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:19.486768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:19.486768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:19.486768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:19.486768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:19.486768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:19.486768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:19.486768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:19.486768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:19.486768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:19.486768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:19.486768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:19.486768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:19.486768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:19.486768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:19.486768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:19.487651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:19.487651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:19.487651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:19.487651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:19.487651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:19.487651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:19.487651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:19.487651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:19.487651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:19.487651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:19.487651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:19.487651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:19.487651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 312/634] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:34:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:19.489032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7ed62-a6be-4c2e-9b83-339590e88b2c None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:19.489032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7ed62-a6be-4c2e-9b83-339590e88b2c None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:19.489032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7ed62-a6be-4c2e-9b83-339590e88b2c None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:19.490203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 313/635] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:19.490693 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac7ed62-a6be-4c2e-9b83-339590e88b2c None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:19.533386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711c5e5-120c-4d81-8c4f-6bd50cc0c5aa None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:19.570136 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376ba00-e112-4833-a5f9-e84c1893f5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:19.571069 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376ba00-e112-4833-a5f9-e84c1893f5e4 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:19.571355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376ba00-e112-4833-a5f9-e84c1893f5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:19.571629 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376ba00-e112-4833-a5f9-e84c1893f5e4 None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:19.572472 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0376ba00-e112-4833-a5f9-e84c1893f5e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:19.572825 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0376ba00-e112-4833-a5f9-e84c1893f5e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:19.574250 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0376ba00-e112-4833-a5f9-e84c1893f5e4 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:19.585661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 323/636] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:19.591715 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9c363-207e-4fad-9372-cc0962f68070 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:19.592002 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9c363-207e-4fad-9372-cc0962f68070 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:19.592283 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9c363-207e-4fad-9372-cc0962f68070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:19.594907 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac7ed62-a6be-4c2e-9b83-339590e88b2c None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:19.608119 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-93c9c363-207e-4fad-9372-cc0962f68070 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:19.609406 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-93c9c363-207e-4fad-9372-cc0962f68070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:19.628042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8686263-e191-4d1e-956b-2f93a5ba9e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:19.628958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8686263-e191-4d1e-956b-2f93a5ba9e72 None admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:19.629229 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8686263-e191-4d1e-956b-2f93a5ba9e72 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:19.629491 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8686263-e191-4d1e-956b-2f93a5ba9e72 None admin] PATH_INFO: `/v3/projects/9cd39464055141b8b43fde20d0885c99/users/380e04358f2349aa912d6a97518877cd/roles/f55c3728df78466989d0cd1b13c5ea28` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:19.643564 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8686263-e191-4d1e-956b-2f93a5ba9e72 None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd39464055141b8b43fde20d0885c99, user_id=380e04358f2349aa912d6a97518877cd, role_id=f55c3728df78466989d0cd1b13c5ea28)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:19.645047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8686263-e191-4d1e-956b-2f93a5ba9e72 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:19.658501 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 314/637] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:34:19 2022] PUT /identity/v3/projects/9cd39464055141b8b43fde20d0885c99/users/380e04358f2349aa912d6a97518877cd/roles/f55c3728df78466989d0cd1b13c5ea28 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:34:19.659276 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8686263-e191-4d1e-956b-2f93a5ba9e72 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:19.681563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 324/638] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:19.681563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9c363-207e-4fad-9372-cc0962f68070 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:19.778953 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8686263-e191-4d1e-956b-2f93a5ba9e72 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:19.781962 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9c363-207e-4fad-9372-cc0962f68070 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:19.813227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641049f-82d5-4c80-8695-2fde9dd6db7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:19.813812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641049f-82d5-4c80-8695-2fde9dd6db7d None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:19.814085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641049f-82d5-4c80-8695-2fde9dd6db7d None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:19.814352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641049f-82d5-4c80-8695-2fde9dd6db7d None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:19.815150 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641049f-82d5-4c80-8695-2fde9dd6db7d None admin] RBAC: Adding query filter params (name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:19.815411 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641049f-82d5-4c80-8695-2fde9dd6db7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:19.816829 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641049f-82d5-4c80-8695-2fde9dd6db7d None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:19.817524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608e7b4-d25c-4edc-932d-44f8642d101c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:19.818439 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608e7b4-d25c-4edc-932d-44f8642d101c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:19.818940 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608e7b4-d25c-4edc-932d-44f8642d101c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:19.819407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608e7b4-d25c-4edc-932d-44f8642d101c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:19.820438 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608e7b4-d25c-4edc-932d-44f8642d101c None admin] RBAC: Adding query filter params (role.id=f55c3728df78466989d0cd1b13c5ea28, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=6362303cb918486c8525225914332ad5) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:19.820979 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608e7b4-d25c-4edc-932d-44f8642d101c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:19.822729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608e7b4-d25c-4edc-932d-44f8642d101c None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:19.825666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 315/639] 10.222.0.26 () {56 vars in 1103 bytes} [Mon May 9 09:34:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:19.827029 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d1656-b501-4b97-bb90-d07351e8ae9c None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:19.827307 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d1656-b501-4b97-bb90-d07351e8ae9c None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:19.827574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d1656-b501-4b97-bb90-d07351e8ae9c None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:19.829027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 316/640] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:19.833661 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d1656-b501-4b97-bb90-d07351e8ae9c None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:19.834935 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 325/641] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:34:19 2022] GET /identity/v3/role_assignments?user.id=6362303cb918486c8525225914332ad5&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=f55c3728df78466989d0cd1b13c5ea28 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:19.835961 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608e7b4-d25c-4edc-932d-44f8642d101c None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:19.955720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d1656-b501-4b97-bb90-d07351e8ae9c None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:19.963196 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608e7b4-d25c-4edc-932d-44f8642d101c None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:19.992874 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed87435-e204-4594-8797-9c01fb7fa07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:19.994088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed87435-e204-4594-8797-9c01fb7fa07c None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:19.994555 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed87435-e204-4594-8797-9c01fb7fa07c None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:19.995060 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed87435-e204-4594-8797-9c01fb7fa07c None admin] PATH_INFO: `/v3/roles/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:20.005979 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d5cd7-f88e-42e0-aafc-73022fa2fbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:20.007059 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d5cd7-f88e-42e0-aafc-73022fa2fbaa None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:20.007340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d5cd7-f88e-42e0-aafc-73022fa2fbaa None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:20.007607 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d5cd7-f88e-42e0-aafc-73022fa2fbaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:20.008031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed87435-e204-4594-8797-9c01fb7fa07c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:20.016525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-bed87435-e204-4594-8797-9c01fb7fa07c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:20.016525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:20.016525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:20.016525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:20.016525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:20.016525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:20.016525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:20.016525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:20.016525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:20.016525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:20.016525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:20.016525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:20.016525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:20.016525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:20.016525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:20.016525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:20.017782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:20.017782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:20.017782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:20.017782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:20.017782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:20.017782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:20.017782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:20.017782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:20.017782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:20.017782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:20.017782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:20.017782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:20.017782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:20.017782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:20.017782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:20.017782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:20.018766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:20.018766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:20.018766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:20.018766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:20.018766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:20.018766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:20.018766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:20.018766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:20.018766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:20.018766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:20.018766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:20.018766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:20.019942 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d5cd7-f88e-42e0-aafc-73022fa2fbaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:20.021296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 317/642] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:34:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:20.021464 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d5cd7-f88e-42e0-aafc-73022fa2fbaa None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:20.023025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f862a-edee-42e4-907b-af61a5a10748 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:20.023457 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f862a-edee-42e4-907b-af61a5a10748 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:20.023879 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f862a-edee-42e4-907b-af61a5a10748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:20.042436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-3b9f862a-edee-42e4-907b-af61a5a10748 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:20.043885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-3b9f862a-edee-42e4-907b-af61a5a10748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:20.048332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-383d5cd7-f88e-42e0-aafc-73022fa2fbaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:20.048332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:20.048332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:20.048332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:20.048332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:20.048332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:20.048332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:20.048332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:20.048332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:20.048332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:20.048332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:20.048332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:20.048332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:20.048332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:20.048332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:20.049414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:20.049414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:20.049414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:20.049414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:20.049414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:20.049414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:20.049414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:20.049414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:20.049414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:20.049414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:20.049414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:20.049414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:20.049414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:20.049414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:20.049414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:20.049414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:20.049414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:20.050896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:20.050896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:20.050896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:20.050896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:20.050896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 326/643] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:20.050896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d5cd7-f88e-42e0-aafc-73022fa2fbaa None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:20.122532 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 318/644] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:20.123888 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f862a-edee-42e4-907b-af61a5a10748 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:20.238307 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f862a-edee-42e4-907b-af61a5a10748 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:20.271862 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0e04d-ad47-420a-a1c6-036df2db2e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:20.272542 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0e04d-ad47-420a-a1c6-036df2db2e87 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:20.272848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0e04d-ad47-420a-a1c6-036df2db2e87 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:20.273121 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0e04d-ad47-420a-a1c6-036df2db2e87 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:20.273846 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc0e04d-ad47-420a-a1c6-036df2db2e87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:20.274144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc0e04d-ad47-420a-a1c6-036df2db2e87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:20.275639 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc0e04d-ad47-420a-a1c6-036df2db2e87 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:20.281761 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d5cd7-f88e-42e0-aafc-73022fa2fbaa None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:20.287327 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 319/645] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:20.288220 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0e04d-ad47-420a-a1c6-036df2db2e87 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:20.315068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c2d53-2bb9-47d2-aebc-50af7c2b5743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:20.315732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c2d53-2bb9-47d2-aebc-50af7c2b5743 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:20.316005 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c2d53-2bb9-47d2-aebc-50af7c2b5743 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:20.316272 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c2d53-2bb9-47d2-aebc-50af7c2b5743 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:20.317061 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754c2d53-2bb9-47d2-aebc-50af7c2b5743 None admin] RBAC: Adding query filter params (name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:20.317360 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754c2d53-2bb9-47d2-aebc-50af7c2b5743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:20.318724 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754c2d53-2bb9-47d2-aebc-50af7c2b5743 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:20.327155 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 327/646] 10.222.0.26 () {56 vars in 1103 bytes} [Mon May 9 09:34:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:20.327981 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c2d53-2bb9-47d2-aebc-50af7c2b5743 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:20.386600 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0e04d-ad47-420a-a1c6-036df2db2e87 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:20.419180 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64536091-aa8d-4aae-87ae-fc252a985efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:20.419985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64536091-aa8d-4aae-87ae-fc252a985efd None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:20.420261 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64536091-aa8d-4aae-87ae-fc252a985efd None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:20.420528 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64536091-aa8d-4aae-87ae-fc252a985efd None admin] PATH_INFO: `/v3/roles/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:20.426641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c2d53-2bb9-47d2-aebc-50af7c2b5743 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:20.433559 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64536091-aa8d-4aae-87ae-fc252a985efd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:20.443741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-64536091-aa8d-4aae-87ae-fc252a985efd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:20.443741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:20.443741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:20.443741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:20.443741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:20.443741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:20.443741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:20.443741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:20.443741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:20.443741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:20.443741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:20.443741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:20.443741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:20.443741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:20.443741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:20.443741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:20.444745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:20.444745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:20.444745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:20.444745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:20.444745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:20.444745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:20.444745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:20.444745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:20.444745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:20.444745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:20.444745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:20.444745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:20.444745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:20.444745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:20.444745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:20.444745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:20.446336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:20.446336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:20.446336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:20.446336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:20.446336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:20.446336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:20.446336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:20.446336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:20.446336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:20.446336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:20.446336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:20.446336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:20.446336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 320/647] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:34:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:20.446336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64536091-aa8d-4aae-87ae-fc252a985efd None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:20.465061 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b6bb5-4704-4ec2-a8fc-fcc34f20b90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:20.466243 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b6bb5-4704-4ec2-a8fc-fcc34f20b90b None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:20.466712 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b6bb5-4704-4ec2-a8fc-fcc34f20b90b None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:20.467170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b6bb5-4704-4ec2-a8fc-fcc34f20b90b None admin] PATH_INFO: `/v3/users/glance` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:20.474589 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b6bb5-4704-4ec2-a8fc-fcc34f20b90b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:20.476386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b6bb5-4704-4ec2-a8fc-fcc34f20b90b None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:20.496309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-1e6b6bb5-4704-4ec2-a8fc-fcc34f20b90b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 09:34:20.496309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:20.496309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:20.496309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:20.496309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:20.496309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:20.496309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:20.496309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:20.496309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:20.496309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:20.496309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:20.496309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:20.496309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:20.496309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:20.496309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:20.496309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:20.497386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:20.497386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:20.497386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:20.497386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:20.497386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:20.497386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:20.497386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:20.497386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:20.497386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:20.497386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:20.497386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:20.497386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:20.497386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:20.497386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:20.497386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:20.497386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:20.497386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:20.497386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:20.499058 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:20.499058 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:20.499058 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:20.499058 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:20.499058 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:20.499058 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:20.499058 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:20.499058 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:20.499058 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 09:34:20.499058 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:20.501984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 328/648] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:34:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:20.502852 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b6bb5-4704-4ec2-a8fc-fcc34f20b90b None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:20.558658 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64536091-aa8d-4aae-87ae-fc252a985efd None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:20.596159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4afbf-444c-4a7f-8181-a62ed5706718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:20.597003 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4afbf-444c-4a7f-8181-a62ed5706718 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:20.597280 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4afbf-444c-4a7f-8181-a62ed5706718 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:20.597543 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4afbf-444c-4a7f-8181-a62ed5706718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:20.611335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4afbf-444c-4a7f-8181-a62ed5706718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:20.613141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4afbf-444c-4a7f-8181-a62ed5706718 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:20.631270 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b6bb5-4704-4ec2-a8fc-fcc34f20b90b None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:20.634946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-22d4afbf-444c-4a7f-8181-a62ed5706718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:20.634946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:20.634946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:20.634946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:20.634946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:20.634946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:20.634946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:20.634946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:20.634946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:20.634946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:20.634946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:20.634946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:20.634946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:20.634946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:20.634946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:20.635837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:20.635837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:20.635837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:20.635837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:20.635837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:20.635837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:20.635837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:20.635837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:20.635837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:20.635837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:20.635837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:20.635837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:20.635837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:20.635837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:20.635837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:20.635837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:20.635837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:20.637476 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:20.637476 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:20.637476 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:20.637476 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:20.637476 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 321/649] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:20.637476 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4afbf-444c-4a7f-8181-a62ed5706718 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:20.669155 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae2fcb-fc38-4d49-929b-5040e561e036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:20.670654 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae2fcb-fc38-4d49-929b-5040e561e036 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:20.671148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae2fcb-fc38-4d49-929b-5040e561e036 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:20.671551 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae2fcb-fc38-4d49-929b-5040e561e036 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:20.672892 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae2fcb-fc38-4d49-929b-5040e561e036 None admin] RBAC: Adding query filter params (name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:20.673355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae2fcb-fc38-4d49-929b-5040e561e036 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:20.675191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae2fcb-fc38-4d49-929b-5040e561e036 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:20.687210 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 329/650] 10.222.0.26 () {56 vars in 1103 bytes} [Mon May 9 09:34:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:20.688587 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae2fcb-fc38-4d49-929b-5040e561e036 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:20.754922 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4afbf-444c-4a7f-8181-a62ed5706718 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:20.795061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a194d-a472-4766-b65d-f72cae1ec837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:20.797199 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a194d-a472-4766-b65d-f72cae1ec837 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:20.797834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a194d-a472-4766-b65d-f72cae1ec837 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:20.798587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a194d-a472-4766-b65d-f72cae1ec837 None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:20.800061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a194d-a472-4766-b65d-f72cae1ec837 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:20.800788 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a194d-a472-4766-b65d-f72cae1ec837 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:20.802822 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a194d-a472-4766-b65d-f72cae1ec837 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:20.808920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae2fcb-fc38-4d49-929b-5040e561e036 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:20.824721 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 322/651] 10.222.0.26 () {56 vars in 1137 bytes} [Mon May 9 09:34:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:20.825815 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a194d-a472-4766-b65d-f72cae1ec837 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:20.850254 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9b900-33d3-4631-b4aa-79e4c6937350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:20.851107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9b900-33d3-4631-b4aa-79e4c6937350 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:20.851551 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9b900-33d3-4631-b4aa-79e4c6937350 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:20.852320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9b900-33d3-4631-b4aa-79e4c6937350 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:20.852782 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9b900-33d3-4631-b4aa-79e4c6937350 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:20.853297 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9b900-33d3-4631-b4aa-79e4c6937350 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:20.854765 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9b900-33d3-4631-b4aa-79e4c6937350 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:20.871214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 330/652] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:20.872259 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9b900-33d3-4631-b4aa-79e4c6937350 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:20.934482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a194d-a472-4766-b65d-f72cae1ec837 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:20.973620 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62637f74-47d3-41a7-bd5c-6503d5413ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:20.974448 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62637f74-47d3-41a7-bd5c-6503d5413ef0 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:20.974822 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62637f74-47d3-41a7-bd5c-6503d5413ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:20.975191 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62637f74-47d3-41a7-bd5c-6503d5413ef0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:20.985417 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9b900-33d3-4631-b4aa-79e4c6937350 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:20.985830 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62637f74-47d3-41a7-bd5c-6503d5413ef0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:20.987386 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62637f74-47d3-41a7-bd5c-6503d5413ef0 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:21.005467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-62637f74-47d3-41a7-bd5c-6503d5413ef0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:21.005467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:21.005467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:21.005467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:21.005467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:21.005467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:21.005467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:21.005467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:21.005467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:21.005467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:21.005467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:21.005467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:21.005467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:21.005467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:21.005467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:21.006585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:21.006585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:21.006585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:21.006585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:21.006585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:21.006585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:21.006585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:21.006585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:21.006585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:21.006585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:21.006585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:21.006585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:21.006585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:21.006585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:21.006585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:21.006585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:21.006585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:21.007677 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:21.007677 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:21.007677 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:21.007677 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:21.009228 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 323/653] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:21.010179 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62637f74-47d3-41a7-bd5c-6503d5413ef0 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:21.021545 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35cb40-e8ea-4afc-b647-381e371489cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:21.022412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35cb40-e8ea-4afc-b647-381e371489cc None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:21.022835 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35cb40-e8ea-4afc-b647-381e371489cc None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:21.023256 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35cb40-e8ea-4afc-b647-381e371489cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:21.086625 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35cb40-e8ea-4afc-b647-381e371489cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:21.088342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35cb40-e8ea-4afc-b647-381e371489cc None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:21.108736 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62637f74-47d3-41a7-bd5c-6503d5413ef0 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:21.144927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c232b-7023-4e95-8e22-97bb9a38de3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:21.145875 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c232b-7023-4e95-8e22-97bb9a38de3c None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:21.146294 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c232b-7023-4e95-8e22-97bb9a38de3c None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:21.146661 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c232b-7023-4e95-8e22-97bb9a38de3c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:21.154401 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c232b-7023-4e95-8e22-97bb9a38de3c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:21.155966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c232b-7023-4e95-8e22-97bb9a38de3c None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:21.171005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-7d0c232b-7023-4e95-8e22-97bb9a38de3c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 09:34:21.171005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:21.171005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:21.171005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:21.171005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:21.171005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:21.171005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:21.171005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:21.171005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:21.171005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:21.171005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:21.171005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:21.171005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:21.171005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:21.171005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:21.171005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:21.172159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:21.172159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:21.172159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:21.172159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:21.172159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:21.172159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:21.172159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:21.172159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:21.172159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:21.172159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:21.172159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:21.172159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:21.172159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:21.172159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:21.172159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:21.172159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:21.172159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:21.172159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:21.173345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:21.173345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:21.173345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:21.173345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:21.173345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:21.173345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:21.173345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:21.173345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:21.173345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 09:34:21.173345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:21.175245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 324/654] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:34:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:21.176222 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c232b-7023-4e95-8e22-97bb9a38de3c None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:21.188367 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 331/655] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:34:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:34:21.189527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35cb40-e8ea-4afc-b647-381e371489cc None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:21.276137 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c232b-7023-4e95-8e22-97bb9a38de3c None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:21.287483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35cb40-e8ea-4afc-b647-381e371489cc None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:21.310290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79644f72-bc8f-446d-8634-21fcfddd2c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:21.311122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79644f72-bc8f-446d-8634-21fcfddd2c94 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:21.311496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79644f72-bc8f-446d-8634-21fcfddd2c94 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:21.311884 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79644f72-bc8f-446d-8634-21fcfddd2c94 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:21.312744 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79644f72-bc8f-446d-8634-21fcfddd2c94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:21.313155 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79644f72-bc8f-446d-8634-21fcfddd2c94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:21.314632 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79644f72-bc8f-446d-8634-21fcfddd2c94 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:21.321478 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c19f673-3066-47fa-829f-68bd257758fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:21.322378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19f673-3066-47fa-829f-68bd257758fb None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:21.322793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19f673-3066-47fa-829f-68bd257758fb None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:21.323212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19f673-3066-47fa-829f-68bd257758fb None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:21.324169 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c19f673-3066-47fa-829f-68bd257758fb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:21.324633 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c19f673-3066-47fa-829f-68bd257758fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:21.326204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c19f673-3066-47fa-829f-68bd257758fb None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:21.327277 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 325/656] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:21.328181 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79644f72-bc8f-446d-8634-21fcfddd2c94 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:21.344146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 332/657] 10.222.0.26 () {56 vars in 1137 bytes} [Mon May 9 09:34:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:21.345283 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c19f673-3066-47fa-829f-68bd257758fb None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:21.427074 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79644f72-bc8f-446d-8634-21fcfddd2c94 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:21.440039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c19f673-3066-47fa-829f-68bd257758fb None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:21.460940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f053373-44a7-40c8-ba09-5b7ca14ca156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:21.461927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f053373-44a7-40c8-ba09-5b7ca14ca156 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:21.462261 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f053373-44a7-40c8-ba09-5b7ca14ca156 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:21.462692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f053373-44a7-40c8-ba09-5b7ca14ca156 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:21.473416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f053373-44a7-40c8-ba09-5b7ca14ca156 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:21.474874 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f053373-44a7-40c8-ba09-5b7ca14ca156 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:21.484270 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9d069-3e46-40c7-b985-0271a34d98ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:21.485418 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9d069-3e46-40c7-b985-0271a34d98ea None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:21.485938 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9d069-3e46-40c7-b985-0271a34d98ea None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:21.486407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9d069-3e46-40c7-b985-0271a34d98ea None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:21.491496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-3f053373-44a7-40c8-ba09-5b7ca14ca156 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:21.491496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:21.491496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:21.491496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:21.491496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:21.491496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:21.491496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:21.491496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:21.491496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:21.491496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:21.491496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:21.491496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:21.491496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:21.491496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:21.491496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:21.492384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:21.492384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:21.492384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:21.492384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:21.492384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:21.492384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:21.492384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:21.492384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:21.492384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:21.492384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:21.492384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:21.492384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:21.492384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:21.492384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:21.492384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:21.492384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:21.492384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:21.494364 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:21.494364 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:21.494364 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:21.494364 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:21.494364 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 326/658] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:21.494364 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f053373-44a7-40c8-ba09-5b7ca14ca156 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:21.499565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9d069-3e46-40c7-b985-0271a34d98ea None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:21.501299 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9d069-3e46-40c7-b985-0271a34d98ea None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:21.519903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-e0b9d069-3e46-40c7-b985-0271a34d98ea None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 09:34:21.519903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:21.519903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:21.519903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:21.519903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:21.519903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:21.519903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:21.519903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:21.519903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:21.519903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:21.519903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:21.519903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:21.519903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:21.519903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:21.519903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:21.519903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:21.520930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:21.520930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:21.520930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:21.520930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:21.520930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:21.520930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:21.520930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:21.520930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:21.520930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:21.520930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:21.520930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:21.520930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:21.520930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:21.520930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:21.520930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:21.520930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:21.520930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:21.520930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:21.521884 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:21.521884 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:21.521884 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:21.521884 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:21.521884 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:21.521884 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:21.521884 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:21.521884 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:21.521884 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 09:34:21.521884 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:21.523822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 333/659] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:34:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:21.523822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9d069-3e46-40c7-b985-0271a34d98ea None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:21.613219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f053373-44a7-40c8-ba09-5b7ca14ca156 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:21.627877 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9d069-3e46-40c7-b985-0271a34d98ea None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:21.655100 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e385ec1b-8c57-4049-b792-9c76fc297e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:21.656543 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385ec1b-8c57-4049-b792-9c76fc297e99 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:21.657087 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385ec1b-8c57-4049-b792-9c76fc297e99 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:21.657500 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385ec1b-8c57-4049-b792-9c76fc297e99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:21.660895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474eb494-999c-4589-946f-ed97375bae41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:21.662222 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474eb494-999c-4589-946f-ed97375bae41 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:21.662222 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474eb494-999c-4589-946f-ed97375bae41 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:21.662222 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474eb494-999c-4589-946f-ed97375bae41 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:21.663251 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474eb494-999c-4589-946f-ed97375bae41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:21.663679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474eb494-999c-4589-946f-ed97375bae41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:21.665191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474eb494-999c-4589-946f-ed97375bae41 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:21.677541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 334/660] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:21.678863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474eb494-999c-4589-946f-ed97375bae41 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:21.737492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e385ec1b-8c57-4049-b792-9c76fc297e99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:21.739165 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e385ec1b-8c57-4049-b792-9c76fc297e99 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:21.779769 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474eb494-999c-4589-946f-ed97375bae41 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:21.829512 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c889b9-2142-4ca6-b6e3-a005b6989a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:21.830311 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c889b9-2142-4ca6-b6e3-a005b6989a03 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:21.830660 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c889b9-2142-4ca6-b6e3-a005b6989a03 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:21.831000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c889b9-2142-4ca6-b6e3-a005b6989a03 None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:21.831911 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c889b9-2142-4ca6-b6e3-a005b6989a03 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:21.832555 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c889b9-2142-4ca6-b6e3-a005b6989a03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:21.833843 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c889b9-2142-4ca6-b6e3-a005b6989a03 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:21.845366 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 327/661] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:34:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:34:21.846600 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e385ec1b-8c57-4049-b792-9c76fc297e99 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:21.852129 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 335/662] 10.222.0.26 () {56 vars in 1139 bytes} [Mon May 9 09:34:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:21.853305 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c889b9-2142-4ca6-b6e3-a005b6989a03 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:21.943132 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e385ec1b-8c57-4049-b792-9c76fc297e99 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:21.950170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c889b9-2142-4ca6-b6e3-a005b6989a03 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:21.976631 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217293e6-e0b6-4a33-be83-f30ecce6180c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:21.977456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217293e6-e0b6-4a33-be83-f30ecce6180c None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:21.977725 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217293e6-e0b6-4a33-be83-f30ecce6180c None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:21.978022 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217293e6-e0b6-4a33-be83-f30ecce6180c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:21.985386 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217293e6-e0b6-4a33-be83-f30ecce6180c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:21.986833 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217293e6-e0b6-4a33-be83-f30ecce6180c None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:21.987720 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657fa90-90a7-4a78-98c8-7d9896534ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:21.988467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657fa90-90a7-4a78-98c8-7d9896534ca8 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:21.988770 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657fa90-90a7-4a78-98c8-7d9896534ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:21.989260 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657fa90-90a7-4a78-98c8-7d9896534ca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:21.999207 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6657fa90-90a7-4a78-98c8-7d9896534ca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:22.000334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-217293e6-e0b6-4a33-be83-f30ecce6180c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:22.000334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:22.000334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:22.000334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:22.000334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:22.000334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:22.000334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:22.000334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:22.000334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:22.000334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:22.000334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:22.000334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:22.000334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:22.000334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:22.000334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:22.001231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:22.001231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:22.001231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:22.001231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:22.001231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:22.001231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:22.001231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:22.001231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:22.001231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:22.001231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:22.001231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:22.001231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:22.001231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:22.001231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:22.001231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:22.001231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:22.001231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:22.002141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:22.002141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:22.002141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:22.002141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:22.002141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:22.002141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:22.002141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:22.002141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:22.002579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6657fa90-90a7-4a78-98c8-7d9896534ca8 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:22.002650 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 328/663] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:22.002650 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217293e6-e0b6-4a33-be83-f30ecce6180c None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:22.022717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-6657fa90-90a7-4a78-98c8-7d9896534ca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:22.022717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:22.022717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:22.022717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:22.022717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:22.022717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:22.022717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:22.022717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:22.022717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:22.022717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:22.022717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:22.022717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:22.022717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:22.022717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:22.022717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:22.023688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:22.023688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:22.023688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:22.023688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:22.023688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:22.023688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:22.023688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:22.023688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:22.023688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:22.023688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:22.023688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:22.023688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:22.023688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:22.023688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:22.023688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:22.023688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:22.023688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:22.024609 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:22.024609 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:22.024609 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:22.024609 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:22.024609 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 336/664] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:22.025808 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716fb00-a4a0-4fc5-a63f-41f3f7ad743d None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:22.025808 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716fb00-a4a0-4fc5-a63f-41f3f7ad743d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:22.025808 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716fb00-a4a0-4fc5-a63f-41f3f7ad743d None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:22.027186 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 337/665] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:22.027884 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716fb00-a4a0-4fc5-a63f-41f3f7ad743d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:22.108592 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217293e6-e0b6-4a33-be83-f30ecce6180c None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:22.136450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716fb00-a4a0-4fc5-a63f-41f3f7ad743d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:22.142553 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382ec27-9410-456c-b0f6-c929b49c69e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:22.143197 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382ec27-9410-456c-b0f6-c929b49c69e9 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:22.143475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382ec27-9410-456c-b0f6-c929b49c69e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:22.143737 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382ec27-9410-456c-b0f6-c929b49c69e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:22.168842 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7418af-ccf7-418d-99fe-c26a05aa30e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:22.169493 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7418af-ccf7-418d-99fe-c26a05aa30e2 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:22.169768 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7418af-ccf7-418d-99fe-c26a05aa30e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:22.170036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7418af-ccf7-418d-99fe-c26a05aa30e2 None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:22.170822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7418af-ccf7-418d-99fe-c26a05aa30e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:22.171121 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7418af-ccf7-418d-99fe-c26a05aa30e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:22.172859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7418af-ccf7-418d-99fe-c26a05aa30e2 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:22.184990 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 338/666] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:22.186272 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7418af-ccf7-418d-99fe-c26a05aa30e2 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:22.214939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382ec27-9410-456c-b0f6-c929b49c69e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:22.216386 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382ec27-9410-456c-b0f6-c929b49c69e9 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:22.287338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7418af-ccf7-418d-99fe-c26a05aa30e2 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:22.312616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 329/667] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:34:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:34:22.313995 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df53410-2acd-4d74-867a-dbc4d1151aad None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:22.314260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df53410-2acd-4d74-867a-dbc4d1151aad None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:22.314558 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df53410-2acd-4d74-867a-dbc4d1151aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:22.320062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec4334-c35c-4303-90e7-69051bbf71bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:22.320738 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec4334-c35c-4303-90e7-69051bbf71bc None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:22.321014 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec4334-c35c-4303-90e7-69051bbf71bc None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:22.321279 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec4334-c35c-4303-90e7-69051bbf71bc None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:22.322029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec4334-c35c-4303-90e7-69051bbf71bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:22.322326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec4334-c35c-4303-90e7-69051bbf71bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:22.323685 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec4334-c35c-4303-90e7-69051bbf71bc None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:22.330597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-6df53410-2acd-4d74-867a-dbc4d1151aad None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:22.331760 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-6df53410-2acd-4d74-867a-dbc4d1151aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:22.335066 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 339/668] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:22.335743 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec4334-c35c-4303-90e7-69051bbf71bc None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:22.394224 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 330/669] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:22.395572 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b52e5-fb9d-4c95-85c4-36fef22c2c2b None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:22.395860 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b52e5-fb9d-4c95-85c4-36fef22c2c2b None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:22.396127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b52e5-fb9d-4c95-85c4-36fef22c2c2b None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:22.400700 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 331/670] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:22.400700 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b52e5-fb9d-4c95-85c4-36fef22c2c2b None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:22.430980 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec4334-c35c-4303-90e7-69051bbf71bc None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:22.463505 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d7720-9a9b-4806-a9f7-99e19cd89adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:22.464148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d7720-9a9b-4806-a9f7-99e19cd89adf None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:22.464420 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d7720-9a9b-4806-a9f7-99e19cd89adf None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:22.464697 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d7720-9a9b-4806-a9f7-99e19cd89adf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:22.465527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d7720-9a9b-4806-a9f7-99e19cd89adf None admin] RBAC: Adding query filter params (role.id=f55c3728df78466989d0cd1b13c5ea28, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=7fd398c547554629927d2a80b3a8dee6) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:22.465831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d7720-9a9b-4806-a9f7-99e19cd89adf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:22.467256 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d7720-9a9b-4806-a9f7-99e19cd89adf None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:22.472795 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 340/671] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:34:22 2022] GET /identity/v3/role_assignments?user.id=7fd398c547554629927d2a80b3a8dee6&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=f55c3728df78466989d0cd1b13c5ea28 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:22.473549 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d7720-9a9b-4806-a9f7-99e19cd89adf None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:22.500108 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b52e5-fb9d-4c95-85c4-36fef22c2c2b None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:22.534547 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46cee9-a774-4d39-a579-192abd1a7e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:22.535358 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46cee9-a774-4d39-a579-192abd1a7e80 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:22.535638 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46cee9-a774-4d39-a579-192abd1a7e80 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:22.535904 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46cee9-a774-4d39-a579-192abd1a7e80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:22.547281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae46cee9-a774-4d39-a579-192abd1a7e80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:22.548727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae46cee9-a774-4d39-a579-192abd1a7e80 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:22.568747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-ae46cee9-a774-4d39-a579-192abd1a7e80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:22.568747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:22.568747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:22.568747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:22.568747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:22.568747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:22.568747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:22.568747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:22.568747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:22.568747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:22.568747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:22.568747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:22.568747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:22.568747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:22.568747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:22.569732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:22.569732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:22.569732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:22.569732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:22.569732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:22.569732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:22.569732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:22.569732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:22.569732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:22.569732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:22.569732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:22.569732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:22.569732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:22.569732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:22.569732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:22.569732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:22.569732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:22.571440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:22.571440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:22.571440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:22.571440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:22.571440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 332/672] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:22.571787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ed967-5ef9-4de0-9b4b-80d2923f6a13 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:22.572182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ed967-5ef9-4de0-9b4b-80d2923f6a13 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:22.572292 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ed967-5ef9-4de0-9b4b-80d2923f6a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:22.589816 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-842ed967-5ef9-4de0-9b4b-80d2923f6a13 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:22.591030 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-842ed967-5ef9-4de0-9b4b-80d2923f6a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:22.598008 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d7720-9a9b-4806-a9f7-99e19cd89adf None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:22.631801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ebc277-3a37-48a3-bf1c-18c8b9e1abe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:22.632613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ebc277-3a37-48a3-bf1c-18c8b9e1abe4 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:22.632919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ebc277-3a37-48a3-bf1c-18c8b9e1abe4 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:22.633192 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ebc277-3a37-48a3-bf1c-18c8b9e1abe4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:22.640777 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ebc277-3a37-48a3-bf1c-18c8b9e1abe4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:22.642245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ebc277-3a37-48a3-bf1c-18c8b9e1abe4 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:22.657111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-48ebc277-3a37-48a3-bf1c-18c8b9e1abe4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:22.657111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:22.657111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:22.657111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:22.657111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:22.657111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:22.657111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:22.657111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:22.657111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:22.657111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:22.657111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:22.657111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:22.657111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:22.657111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:22.657111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:22.658180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:22.658180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:22.658180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:22.658180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:22.658180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:22.658180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:22.658180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:22.658180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:22.658180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:22.658180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:22.658180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:22.658180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:22.658180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:22.658180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:22.658180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:22.658180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:22.658180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:22.661373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:22.661373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:22.661373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:22.661373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:22.661373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:22.661373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:22.661373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:22.661373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:22.661373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 341/673] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:22.661373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ebc277-3a37-48a3-bf1c-18c8b9e1abe4 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:22.661932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 333/674] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:22.662350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ed967-5ef9-4de0-9b4b-80d2923f6a13 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:22.764411 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ed967-5ef9-4de0-9b4b-80d2923f6a13 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:22.772386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ebc277-3a37-48a3-bf1c-18c8b9e1abe4 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:22.799964 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c4f4c-ce9c-4821-9d62-6608d660a8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:22.800877 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c4f4c-ce9c-4821-9d62-6608d660a8ca None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:22.801406 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c4f4c-ce9c-4821-9d62-6608d660a8ca None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:22.801674 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c4f4c-ce9c-4821-9d62-6608d660a8ca None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:22.802382 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c4f4c-ce9c-4821-9d62-6608d660a8ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:22.802681 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c4f4c-ce9c-4821-9d62-6608d660a8ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:22.804044 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c4f4c-ce9c-4821-9d62-6608d660a8ca None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:22.804921 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2220cbb-2bce-44c5-9c2b-1eaec340d43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:22.805714 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2220cbb-2bce-44c5-9c2b-1eaec340d43a None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:22.805987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2220cbb-2bce-44c5-9c2b-1eaec340d43a None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:22.806291 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2220cbb-2bce-44c5-9c2b-1eaec340d43a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:22.815932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 334/675] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:22.817265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c4f4c-ce9c-4821-9d62-6608d660a8ca None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:22.817514 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2220cbb-2bce-44c5-9c2b-1eaec340d43a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:22.825240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-b2220cbb-2bce-44c5-9c2b-1eaec340d43a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:22.825240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:22.825240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:22.825240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:22.825240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:22.825240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:22.825240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:22.825240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:22.825240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:22.825240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:22.825240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:22.825240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:22.825240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:22.825240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:22.825240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:22.825240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:22.826347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:22.826347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:22.826347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:22.826347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:22.826347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:22.826347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:22.826347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:22.826347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:22.826347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:22.826347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:22.826347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:22.826347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:22.826347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:22.826347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:22.826347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:22.826347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:22.827199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:22.827199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:22.827199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:22.827199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:22.827199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:22.827199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:22.827199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:22.827199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:22.827199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:22.827199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:22.827199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:22.827199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:22.827199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 342/676] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:34:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:22.827199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2220cbb-2bce-44c5-9c2b-1eaec340d43a None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:22.917025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c4f4c-ce9c-4821-9d62-6608d660a8ca None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:22.930475 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2220cbb-2bce-44c5-9c2b-1eaec340d43a None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:22.950531 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04289f86-030e-4208-ae36-ca54154ab8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:22.951183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04289f86-030e-4208-ae36-ca54154ab8a8 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:22.951454 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04289f86-030e-4208-ae36-ca54154ab8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:22.951719 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04289f86-030e-4208-ae36-ca54154ab8a8 None admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:22.952502 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04289f86-030e-4208-ae36-ca54154ab8a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:22.952832 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04289f86-030e-4208-ae36-ca54154ab8a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:22.954271 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04289f86-030e-4208-ae36-ca54154ab8a8 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:22.963801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f79b0-ae71-472b-930b-b28d2a129200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:22.964592 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f79b0-ae71-472b-930b-b28d2a129200 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:22.964902 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f79b0-ae71-472b-930b-b28d2a129200 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:22.965168 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f79b0-ae71-472b-930b-b28d2a129200 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:22.970602 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 335/677] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:22.970602 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04289f86-030e-4208-ae36-ca54154ab8a8 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:22.979811 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f79b0-ae71-472b-930b-b28d2a129200 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:22.987429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-ef7f79b0-ae71-472b-930b-b28d2a129200 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:34:22.987429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:22.987429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:22.987429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:22.987429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:22.987429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:22.987429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:22.987429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:22.987429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:22.987429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:22.987429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:22.987429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:22.987429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:22.987429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:22.987429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:22.987429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:22.988341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:22.988341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:22.988341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:22.988341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:22.988341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:22.988341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:22.988341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:22.988341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:22.988341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:22.988341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:22.988341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:22.988341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:22.988341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:22.988341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:22.988341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:22.988341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:22.989571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:22.989571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:22.989571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:22.989571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:22.989571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:22.989571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:22.989571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:22.989571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:22.989571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:22.989571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:22.989571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:34:22.989571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:22.989571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 343/678] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:34:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:22.989571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f79b0-ae71-472b-930b-b28d2a129200 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:23.063668 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04289f86-030e-4208-ae36-ca54154ab8a8 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:23.086660 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f79b0-ae71-472b-930b-b28d2a129200 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:23.096645 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c13c5-c9d4-4df3-9130-195a7dcdd8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:23.097460 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c13c5-c9d4-4df3-9130-195a7dcdd8ef None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:23.097736 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c13c5-c9d4-4df3-9130-195a7dcdd8ef None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:23.098016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c13c5-c9d4-4df3-9130-195a7dcdd8ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:23.104535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c13c5-c9d4-4df3-9130-195a7dcdd8ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:23.106017 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c13c5-c9d4-4df3-9130-195a7dcdd8ef None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:23.117802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e5585-ad4b-458a-a9df-5359f7415ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:23.118460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e5585-ad4b-458a-a9df-5359f7415ac3 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:23.118807 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e5585-ad4b-458a-a9df-5359f7415ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:23.119043 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e5585-ad4b-458a-a9df-5359f7415ac3 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:23.119618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-984c13c5-c9d4-4df3-9130-195a7dcdd8ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:23.119618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:23.119618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:23.119618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:23.119618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:23.119618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:23.119618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:23.119618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:23.119618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:23.119618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:23.119618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:23.119618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:23.119618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:23.119618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:23.119618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:23.121032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:23.121032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:23.121032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:23.121032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:23.121032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:23.121032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:23.121032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:23.121032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:23.121032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:23.121032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:23.121032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:23.121032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:23.121032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:23.121032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:23.121032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:23.121032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:23.121032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:23.126779 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:23.126779 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:23.126779 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:23.126779 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:23.126779 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:23.126779 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:23.126779 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:23.126779 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:23.127229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e5585-ad4b-458a-a9df-5359f7415ac3 None admin] RBAC: Adding query filter params (name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:23.127229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e5585-ad4b-458a-a9df-5359f7415ac3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:23.127356 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 336/679] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:23.127423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e5585-ad4b-458a-a9df-5359f7415ac3 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:23.127491 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c13c5-c9d4-4df3-9130-195a7dcdd8ef None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:23.131389 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 344/680] 10.222.0.26 () {56 vars in 1103 bytes} [Mon May 9 09:34:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:23.131585 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e5585-ad4b-458a-a9df-5359f7415ac3 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:23.228293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e5585-ad4b-458a-a9df-5359f7415ac3 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:23.232904 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c13c5-c9d4-4df3-9130-195a7dcdd8ef None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:23.269573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f834eda-5c66-499d-881e-2255f60c2192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:23.270661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f834eda-5c66-499d-881e-2255f60c2192 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:23.271145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f834eda-5c66-499d-881e-2255f60c2192 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:23.271616 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f834eda-5c66-499d-881e-2255f60c2192 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:23.272570 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f834eda-5c66-499d-881e-2255f60c2192 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:23.272980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18de2db-7158-47ec-8e51-08b987b0351e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:23.273232 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f834eda-5c66-499d-881e-2255f60c2192 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:23.273633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18de2db-7158-47ec-8e51-08b987b0351e None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:23.273911 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18de2db-7158-47ec-8e51-08b987b0351e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:23.274182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18de2db-7158-47ec-8e51-08b987b0351e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:23.275071 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18de2db-7158-47ec-8e51-08b987b0351e None admin] RBAC: Adding query filter params (role.id=f55c3728df78466989d0cd1b13c5ea28, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=a44d689993bd4d30858a770012ee3e89) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:23.275378 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18de2db-7158-47ec-8e51-08b987b0351e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:23.275688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f834eda-5c66-499d-881e-2255f60c2192 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:23.276831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18de2db-7158-47ec-8e51-08b987b0351e None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:23.282625 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 337/681] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:34:23 2022] GET /identity/v3/role_assignments?user.id=a44d689993bd4d30858a770012ee3e89&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=f55c3728df78466989d0cd1b13c5ea28 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:23.283425 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18de2db-7158-47ec-8e51-08b987b0351e None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:23.284641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 345/682] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:34:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:23.285617 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f834eda-5c66-499d-881e-2255f60c2192 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:23.400129 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f834eda-5c66-499d-881e-2255f60c2192 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:23.400857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18de2db-7158-47ec-8e51-08b987b0351e None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:23.434665 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7896b8-715d-4fc6-9111-a6fb247fd1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:23.435454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7896b8-715d-4fc6-9111-a6fb247fd1c0 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:23.435729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7896b8-715d-4fc6-9111-a6fb247fd1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:23.435948 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a095d14-7755-4f68-9bdd-b3d59c601a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:23.436073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7896b8-715d-4fc6-9111-a6fb247fd1c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:23.436908 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a095d14-7755-4f68-9bdd-b3d59c601a3a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:23.437333 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a095d14-7755-4f68-9bdd-b3d59c601a3a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:23.437747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a095d14-7755-4f68-9bdd-b3d59c601a3a None admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:23.438686 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a095d14-7755-4f68-9bdd-b3d59c601a3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:23.439137 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a095d14-7755-4f68-9bdd-b3d59c601a3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:23.440804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a095d14-7755-4f68-9bdd-b3d59c601a3a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:23.448092 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7896b8-715d-4fc6-9111-a6fb247fd1c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:23.451522 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7896b8-715d-4fc6-9111-a6fb247fd1c0 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:23.457005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 338/683] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:23.458138 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a095d14-7755-4f68-9bdd-b3d59c601a3a None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:23.469944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-dd7896b8-715d-4fc6-9111-a6fb247fd1c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:23.469944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:23.469944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:23.469944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:23.469944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:23.469944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:23.469944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:23.469944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:23.469944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:23.469944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:23.469944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:23.469944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:23.469944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:23.469944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:23.469944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:23.470920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:23.470920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:23.470920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:23.470920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:23.470920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:23.470920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:23.470920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:23.470920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:23.470920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:23.470920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:23.470920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:23.470920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:23.470920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:23.470920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:23.470920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:23.470920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:23.470920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:23.471829 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:23.471829 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:23.471829 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:23.471829 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:23.476062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 346/684] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:23.477024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7896b8-715d-4fc6-9111-a6fb247fd1c0 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:23.584014 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a095d14-7755-4f68-9bdd-b3d59c601a3a None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:23.609978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7896b8-715d-4fc6-9111-a6fb247fd1c0 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:23.630960 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d7c458-abe4-43e3-9593-2a5acd2abb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:23.631753 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7c458-abe4-43e3-9593-2a5acd2abb0e None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:23.632170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7c458-abe4-43e3-9593-2a5acd2abb0e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:23.632444 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7c458-abe4-43e3-9593-2a5acd2abb0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:23.643133 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e261e6-f4f2-40b5-9bca-fb6af74802d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:23.643965 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e261e6-f4f2-40b5-9bca-fb6af74802d7 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:23.644339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e261e6-f4f2-40b5-9bca-fb6af74802d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:23.644728 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e261e6-f4f2-40b5-9bca-fb6af74802d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:23.645182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d7c458-abe4-43e3-9593-2a5acd2abb0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:23.645850 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e261e6-f4f2-40b5-9bca-fb6af74802d7 None admin] RBAC: Adding query filter params (role.id=f55c3728df78466989d0cd1b13c5ea28, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=0f0d3f4264a04093894d6bc5c5c45c7c) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:23.646261 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e261e6-f4f2-40b5-9bca-fb6af74802d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:23.646631 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d7c458-abe4-43e3-9593-2a5acd2abb0e None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:23.647863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e261e6-f4f2-40b5-9bca-fb6af74802d7 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:23.654382 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 347/685] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:34:23 2022] GET /identity/v3/role_assignments?user.id=0f0d3f4264a04093894d6bc5c5c45c7c&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=f55c3728df78466989d0cd1b13c5ea28 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:23.655278 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e261e6-f4f2-40b5-9bca-fb6af74802d7 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:23.681616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-a6d7c458-abe4-43e3-9593-2a5acd2abb0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:23.681616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:23.681616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:23.681616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:23.681616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:23.681616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:23.681616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:23.681616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:23.681616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:23.681616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:23.681616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:23.681616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:23.681616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:23.681616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:23.681616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:23.684287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:23.684287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:23.684287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:23.684287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:23.684287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:23.684287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:23.684287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:23.684287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:23.684287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:23.684287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:23.684287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:23.684287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:23.684287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:23.684287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:23.684287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:23.684287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:23.684287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:23.686225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:23.686225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:23.686225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:23.686225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:23.686225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 339/686] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:23.690062 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d7c458-abe4-43e3-9593-2a5acd2abb0e None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:23.811795 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e261e6-f4f2-40b5-9bca-fb6af74802d7 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:23.843968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d7c458-abe4-43e3-9593-2a5acd2abb0e None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:23.846929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37777bb7-ba49-4213-8fe9-c8bbf3da1319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:23.847852 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37777bb7-ba49-4213-8fe9-c8bbf3da1319 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:23.848327 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37777bb7-ba49-4213-8fe9-c8bbf3da1319 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:23.848823 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37777bb7-ba49-4213-8fe9-c8bbf3da1319 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:23.849752 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37777bb7-ba49-4213-8fe9-c8bbf3da1319 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:23.850272 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37777bb7-ba49-4213-8fe9-c8bbf3da1319 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:23.851833 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37777bb7-ba49-4213-8fe9-c8bbf3da1319 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:23.871682 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 348/687] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:23.877578 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37777bb7-ba49-4213-8fe9-c8bbf3da1319 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:23.939848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee6ac5-ed4c-4fc1-9b5b-2de87397c8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:23.939848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee6ac5-ed4c-4fc1-9b5b-2de87397c8ee None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:23.940143 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee6ac5-ed4c-4fc1-9b5b-2de87397c8ee None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:23.940375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee6ac5-ed4c-4fc1-9b5b-2de87397c8ee None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:23.941103 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee6ac5-ed4c-4fc1-9b5b-2de87397c8ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:23.941412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee6ac5-ed4c-4fc1-9b5b-2de87397c8ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:23.942767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee6ac5-ed4c-4fc1-9b5b-2de87397c8ee None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:23.960552 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 340/688] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:23.961901 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee6ac5-ed4c-4fc1-9b5b-2de87397c8ee None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:23.988263 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37777bb7-ba49-4213-8fe9-c8bbf3da1319 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:24.023360 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc44eb-4c53-480a-b730-e33a7a718d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:24.024480 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc44eb-4c53-480a-b730-e33a7a718d65 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:24.025102 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc44eb-4c53-480a-b730-e33a7a718d65 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:24.025612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc44eb-4c53-480a-b730-e33a7a718d65 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:24.032982 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc44eb-4c53-480a-b730-e33a7a718d65 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:24.034657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc44eb-4c53-480a-b730-e33a7a718d65 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:24.049513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-75fc44eb-4c53-480a-b730-e33a7a718d65 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 09:34:24.049513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:24.049513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:24.049513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:24.049513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:24.049513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:24.049513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:24.049513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:24.049513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:24.049513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:24.049513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:24.049513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:24.049513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:24.049513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:24.049513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:24.049513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:24.050619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:24.050619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:24.050619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:24.050619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:24.050619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:24.050619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:24.050619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:24.050619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:24.050619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:24.050619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:24.050619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:24.050619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:24.050619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:24.050619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:24.050619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:24.050619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:24.050619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:24.050619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:24.051593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:24.051593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:24.051593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:24.051593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:24.051593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:24.051593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:24.051593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:24.051593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:24.051593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 09:34:24.051593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:24.054278 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 349/689] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:34:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:24.059910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc44eb-4c53-480a-b730-e33a7a718d65 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:24.062271 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee6ac5-ed4c-4fc1-9b5b-2de87397c8ee None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:24.096138 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e50a2-4be8-4a4a-8795-fc73e06891e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:24.096547 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e50a2-4be8-4a4a-8795-fc73e06891e9 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:24.096858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e50a2-4be8-4a4a-8795-fc73e06891e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:24.097124 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e50a2-4be8-4a4a-8795-fc73e06891e9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:24.105554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e50a2-4be8-4a4a-8795-fc73e06891e9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:24.107002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e50a2-4be8-4a4a-8795-fc73e06891e9 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:24.121225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-5b2e50a2-4be8-4a4a-8795-fc73e06891e9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 09:34:24.121225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:24.121225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:24.121225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:24.121225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:24.121225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:24.121225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:24.121225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:24.121225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:24.121225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:24.121225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:24.121225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:24.121225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:24.121225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:24.121225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:24.121225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:24.122190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:24.122190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:24.122190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:24.122190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:24.122190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:24.122190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:24.122190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:24.122190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:24.122190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:24.122190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:24.122190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:24.122190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:24.122190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:24.122190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:24.122190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:24.122190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:24.122190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:24.122190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:24.123156 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:24.123156 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:24.123156 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:24.123156 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:24.123156 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:24.123156 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:24.123156 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:24.123156 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:24.123156 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 09:34:24.123156 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:24.126853 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 341/690] 10.222.0.26 () {56 vars in 1093 bytes} [Mon May 9 09:34:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:24.128094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e50a2-4be8-4a4a-8795-fc73e06891e9 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:24.165094 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc44eb-4c53-480a-b730-e33a7a718d65 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:24.200314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d3311-3715-4388-a9fa-40fa96117190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:24.201345 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d3311-3715-4388-a9fa-40fa96117190 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:24.201827 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d3311-3715-4388-a9fa-40fa96117190 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:24.202314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d3311-3715-4388-a9fa-40fa96117190 None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:24.203366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d3311-3715-4388-a9fa-40fa96117190 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:24.203888 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d3311-3715-4388-a9fa-40fa96117190 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:24.206299 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d3311-3715-4388-a9fa-40fa96117190 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:24.224593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 350/691] 10.222.0.26 () {56 vars in 1135 bytes} [Mon May 9 09:34:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:24.227647 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e50a2-4be8-4a4a-8795-fc73e06891e9 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:24.230117 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d3311-3715-4388-a9fa-40fa96117190 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:24.261642 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef5c41-8011-4c35-87da-eab824ac36b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:24.262343 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef5c41-8011-4c35-87da-eab824ac36b3 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:24.262682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef5c41-8011-4c35-87da-eab824ac36b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:24.262954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef5c41-8011-4c35-87da-eab824ac36b3 None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:24.263905 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef5c41-8011-4c35-87da-eab824ac36b3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:24.264084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef5c41-8011-4c35-87da-eab824ac36b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:24.266345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef5c41-8011-4c35-87da-eab824ac36b3 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:24.289709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 342/692] 10.222.0.26 () {56 vars in 1133 bytes} [Mon May 9 09:34:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:24.291088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef5c41-8011-4c35-87da-eab824ac36b3 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:24.334165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d3311-3715-4388-a9fa-40fa96117190 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:24.368100 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7891063d-89a7-41aa-9a40-9a222714a107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:24.369160 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891063d-89a7-41aa-9a40-9a222714a107 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:24.369692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891063d-89a7-41aa-9a40-9a222714a107 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:24.370190 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891063d-89a7-41aa-9a40-9a222714a107 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:24.395924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef5c41-8011-4c35-87da-eab824ac36b3 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:24.436810 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7891063d-89a7-41aa-9a40-9a222714a107 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:24.438951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7891063d-89a7-41aa-9a40-9a222714a107 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:24.453723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ef4e5-7b35-4ac2-a33a-f537ac0f0e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:24.454358 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ef4e5-7b35-4ac2-a33a-f537ac0f0e11 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:24.454859 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ef4e5-7b35-4ac2-a33a-f537ac0f0e11 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:24.454859 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ef4e5-7b35-4ac2-a33a-f537ac0f0e11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:24.542710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 351/693] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:34:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:34:24.548598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7891063d-89a7-41aa-9a40-9a222714a107 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:24.581566 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ef4e5-7b35-4ac2-a33a-f537ac0f0e11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:24.583062 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ef4e5-7b35-4ac2-a33a-f537ac0f0e11 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:24.660076 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7891063d-89a7-41aa-9a40-9a222714a107 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:24.697476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732aacd-c302-49d3-b3cb-74081e44de10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:24.698842 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732aacd-c302-49d3-b3cb-74081e44de10 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:24.699320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732aacd-c302-49d3-b3cb-74081e44de10 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:24.699805 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732aacd-c302-49d3-b3cb-74081e44de10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:24.711795 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3732aacd-c302-49d3-b3cb-74081e44de10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:24.713690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3732aacd-c302-49d3-b3cb-74081e44de10 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:24.733973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-3732aacd-c302-49d3-b3cb-74081e44de10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:24.733973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:24.733973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:24.733973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:24.733973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:24.733973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:24.733973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:24.733973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:24.733973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:24.733973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:24.733973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:24.733973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:24.733973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:24.733973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:24.733973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:24.734947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:24.734947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:24.734947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:24.734947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:24.734947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:24.734947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:24.734947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:24.734947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:24.734947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:24.734947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:24.734947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:24.734947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:24.734947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:24.734947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:24.734947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:24.734947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:24.734947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:24.735896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:24.735896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:24.735896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:24.735896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:24.738564 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 352/694] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:24.743822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732aacd-c302-49d3-b3cb-74081e44de10 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:24.843772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 343/695] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:34:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:34:24.849190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ef4e5-7b35-4ac2-a33a-f537ac0f0e11 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:24.855335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732aacd-c302-49d3-b3cb-74081e44de10 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:24.891032 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270ba12-61db-4ec5-a93a-704c5185cb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:24.892477 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270ba12-61db-4ec5-a93a-704c5185cb95 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:24.893006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270ba12-61db-4ec5-a93a-704c5185cb95 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:24.893488 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270ba12-61db-4ec5-a93a-704c5185cb95 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:24.894423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270ba12-61db-4ec5-a93a-704c5185cb95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:24.894922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270ba12-61db-4ec5-a93a-704c5185cb95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:24.896484 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270ba12-61db-4ec5-a93a-704c5185cb95 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:24.909541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 353/696] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:24.917688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270ba12-61db-4ec5-a93a-704c5185cb95 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:24.993672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ef4e5-7b35-4ac2-a33a-f537ac0f0e11 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:25.027247 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449afb4d-ed33-4e6e-bbcb-e7d0c587aa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:25.027873 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449afb4d-ed33-4e6e-bbcb-e7d0c587aa5a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:25.028153 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449afb4d-ed33-4e6e-bbcb-e7d0c587aa5a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:25.028427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449afb4d-ed33-4e6e-bbcb-e7d0c587aa5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:25.040636 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449afb4d-ed33-4e6e-bbcb-e7d0c587aa5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:25.040814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270ba12-61db-4ec5-a93a-704c5185cb95 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:25.042143 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449afb4d-ed33-4e6e-bbcb-e7d0c587aa5a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:25.064225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-449afb4d-ed33-4e6e-bbcb-e7d0c587aa5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:25.064225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:25.064225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:25.064225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:25.064225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:25.064225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:25.064225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:25.064225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:25.064225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:25.064225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:25.064225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:25.064225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:25.064225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:25.064225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:25.064225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:25.065219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:25.065219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:25.065219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:25.065219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:25.065219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:25.065219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:25.065219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:25.065219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:25.065219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:25.065219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:25.065219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:25.065219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:25.065219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:25.065219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:25.065219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:25.065219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:25.065219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:25.066130 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:25.066130 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:25.066130 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:25.066130 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:25.066130 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 344/697] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:25.067166 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefedf1-3d24-4dfb-81a1-c3674f4c3893 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:25.067446 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefedf1-3d24-4dfb-81a1-c3674f4c3893 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:25.067720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefedf1-3d24-4dfb-81a1-c3674f4c3893 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:25.072727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 345/698] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:25.073759 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefedf1-3d24-4dfb-81a1-c3674f4c3893 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:25.084963 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06139c-4e12-4662-b35f-0c01b3b5e3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:25.086552 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06139c-4e12-4662-b35f-0c01b3b5e3f8 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:25.087051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06139c-4e12-4662-b35f-0c01b3b5e3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:25.087896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06139c-4e12-4662-b35f-0c01b3b5e3f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:25.096868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb06139c-4e12-4662-b35f-0c01b3b5e3f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:25.098542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb06139c-4e12-4662-b35f-0c01b3b5e3f8 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:25.116557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-cb06139c-4e12-4662-b35f-0c01b3b5e3f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:25.116557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:25.116557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:25.116557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:25.116557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:25.116557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:25.116557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:25.116557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:25.116557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:25.116557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:25.116557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:25.116557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:25.116557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:25.116557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:25.116557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:25.117527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:25.117527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:25.117527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:25.117527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:25.117527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:25.117527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:25.117527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:25.117527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:25.117527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:25.117527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:25.117527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:25.117527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:25.117527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:25.117527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:25.117527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:25.117527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:25.117527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:25.118431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:25.118431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:25.118431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:25.118431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:25.118431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:25.118431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:25.118431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:25.118431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:25.122139 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 354/699] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:25.123947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f386fa9-91c4-4941-b70a-98976bd5ae07 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:25.124416 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f386fa9-91c4-4941-b70a-98976bd5ae07 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:25.124894 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f386fa9-91c4-4941-b70a-98976bd5ae07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:25.145864 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-7f386fa9-91c4-4941-b70a-98976bd5ae07 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:25.147574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-7f386fa9-91c4-4941-b70a-98976bd5ae07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:25.193910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefedf1-3d24-4dfb-81a1-c3674f4c3893 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:25.225003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 355/700] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:25.226007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f386fa9-91c4-4941-b70a-98976bd5ae07 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:25.233744 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc94a1-957a-4fbe-914c-bfac44ba1b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:25.234672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc94a1-957a-4fbe-914c-bfac44ba1b4a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:25.235128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc94a1-957a-4fbe-914c-bfac44ba1b4a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:25.235584 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc94a1-957a-4fbe-914c-bfac44ba1b4a None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:25.236479 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc94a1-957a-4fbe-914c-bfac44ba1b4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:25.236982 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc94a1-957a-4fbe-914c-bfac44ba1b4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:25.238507 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc94a1-957a-4fbe-914c-bfac44ba1b4a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:25.252754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 346/701] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:25.264744 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc94a1-957a-4fbe-914c-bfac44ba1b4a None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:25.353973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f386fa9-91c4-4941-b70a-98976bd5ae07 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:25.383642 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc94a1-957a-4fbe-914c-bfac44ba1b4a None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:25.388881 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f0784-cf11-460b-9e5a-38b84a17849c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:25.389619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f0784-cf11-460b-9e5a-38b84a17849c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:25.389972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f0784-cf11-460b-9e5a-38b84a17849c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:25.390304 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f0784-cf11-460b-9e5a-38b84a17849c None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:25.391171 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f0784-cf11-460b-9e5a-38b84a17849c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:25.391571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f0784-cf11-460b-9e5a-38b84a17849c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:25.393120 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f0784-cf11-460b-9e5a-38b84a17849c None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:25.405299 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 356/702] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:25.406379 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f0784-cf11-460b-9e5a-38b84a17849c None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:25.424938 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba9330-11f9-47dc-a977-760e638c7b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:25.426316 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba9330-11f9-47dc-a977-760e638c7b7f None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:25.426735 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba9330-11f9-47dc-a977-760e638c7b7f None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:25.427142 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba9330-11f9-47dc-a977-760e638c7b7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:25.437976 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba9330-11f9-47dc-a977-760e638c7b7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:25.440051 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba9330-11f9-47dc-a977-760e638c7b7f None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:25.459002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-a1ba9330-11f9-47dc-a977-760e638c7b7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:25.459002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:25.459002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:25.459002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:25.459002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:25.459002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:25.459002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:25.459002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:25.459002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:25.459002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:25.459002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:25.459002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:25.459002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:25.459002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:25.459002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:25.460162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:25.460162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:25.460162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:25.460162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:25.460162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:25.460162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:25.460162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:25.460162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:25.460162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:25.460162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:25.460162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:25.460162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:25.460162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:25.460162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:25.460162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:25.460162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:25.460162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:25.461582 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:25.461582 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:25.461582 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:25.461582 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:25.461582 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:25.461582 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:25.461582 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:25.461582 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:25.469010 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 347/703] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:25.469010 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba9330-11f9-47dc-a977-760e638c7b7f None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:25.519428 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f0784-cf11-460b-9e5a-38b84a17849c None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:25.571667 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13dc68-f6d1-4c9d-a953-f4ef91ab09cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:25.572938 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13dc68-f6d1-4c9d-a953-f4ef91ab09cd None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:25.573365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13dc68-f6d1-4c9d-a953-f4ef91ab09cd None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:25.574056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13dc68-f6d1-4c9d-a953-f4ef91ab09cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:25.579078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba9330-11f9-47dc-a977-760e638c7b7f None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:25.587497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13dc68-f6d1-4c9d-a953-f4ef91ab09cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:25.589547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13dc68-f6d1-4c9d-a953-f4ef91ab09cd None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:25.611694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-ea13dc68-f6d1-4c9d-a953-f4ef91ab09cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:25.611694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:25.611694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:25.611694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:25.611694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:25.611694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:25.611694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:25.611694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:25.611694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:25.611694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:25.611694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:25.611694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:25.611694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:25.611694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:25.611694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:25.612908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:25.612908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:25.612908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:25.612908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:25.612908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:25.612908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:25.612908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:25.612908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:25.612908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:25.612908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:25.612908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:25.612908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:25.612908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:25.612908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:25.612908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:25.612908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:25.612908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:25.614042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:25.614042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:25.614042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:25.614042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:25.614488 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ede0d-d9ab-4e20-8111-3ee8daf61a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:25.615274 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ede0d-d9ab-4e20-8111-3ee8daf61a0c None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:25.615636 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ede0d-d9ab-4e20-8111-3ee8daf61a0c None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:25.615991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ede0d-d9ab-4e20-8111-3ee8daf61a0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:25.616533 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 357/704] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:25.617101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ede0d-d9ab-4e20-8111-3ee8daf61a0c None admin] RBAC: Adding query filter params (role.id=f55c3728df78466989d0cd1b13c5ea28, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=380e04358f2349aa912d6a97518877cd) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:25.617560 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ede0d-d9ab-4e20-8111-3ee8daf61a0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:25.617869 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13dc68-f6d1-4c9d-a953-f4ef91ab09cd None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:25.619299 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ede0d-d9ab-4e20-8111-3ee8daf61a0c None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:25.627490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 348/705] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:34:25 2022] GET /identity/v3/role_assignments?user.id=380e04358f2349aa912d6a97518877cd&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=f55c3728df78466989d0cd1b13c5ea28 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:25.628533 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ede0d-d9ab-4e20-8111-3ee8daf61a0c None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:25.726432 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13dc68-f6d1-4c9d-a953-f4ef91ab09cd None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:25.732119 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ede0d-d9ab-4e20-8111-3ee8daf61a0c None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:25.762001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc770cc5-d5f8-4ce3-9d02-3fb94dcf679c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:25.763383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc770cc5-d5f8-4ce3-9d02-3fb94dcf679c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:25.763842 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc770cc5-d5f8-4ce3-9d02-3fb94dcf679c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:25.764492 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc770cc5-d5f8-4ce3-9d02-3fb94dcf679c None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:25.765467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc770cc5-d5f8-4ce3-9d02-3fb94dcf679c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:25.765916 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc770cc5-d5f8-4ce3-9d02-3fb94dcf679c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:25.767489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc770cc5-d5f8-4ce3-9d02-3fb94dcf679c None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:25.768484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bfd0a-6e5c-4213-a3f7-b6d78aa4c982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:25.769292 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bfd0a-6e5c-4213-a3f7-b6d78aa4c982 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:25.769724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bfd0a-6e5c-4213-a3f7-b6d78aa4c982 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:25.770027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bfd0a-6e5c-4213-a3f7-b6d78aa4c982 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:25.771106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38bfd0a-6e5c-4213-a3f7-b6d78aa4c982 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:25.771313 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38bfd0a-6e5c-4213-a3f7-b6d78aa4c982 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:25.772836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38bfd0a-6e5c-4213-a3f7-b6d78aa4c982 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:25.783368 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 358/706] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:25.785806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8117dc-4de3-4379-8c07-6a489799582c None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:25.786226 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8117dc-4de3-4379-8c07-6a489799582c None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:25.786655 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8117dc-4de3-4379-8c07-6a489799582c None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:25.788412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 359/707] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:25.791647 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8117dc-4de3-4379-8c07-6a489799582c None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:25.793586 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 349/708] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:25.795109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dde01c-7edd-45bc-b693-e6718b8b8700 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:25.795487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dde01c-7edd-45bc-b693-e6718b8b8700 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:25.795857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dde01c-7edd-45bc-b693-e6718b8b8700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:25.813552 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-30dde01c-7edd-45bc-b693-e6718b8b8700 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:25.815077 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-30dde01c-7edd-45bc-b693-e6718b8b8700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:25.884685 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 350/709] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:25.885803 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dde01c-7edd-45bc-b693-e6718b8b8700 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:25.919879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8117dc-4de3-4379-8c07-6a489799582c None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:25.962470 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42935b1f-03fc-47a2-9cda-8644434e2e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:25.963508 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42935b1f-03fc-47a2-9cda-8644434e2e43 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:25.963980 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42935b1f-03fc-47a2-9cda-8644434e2e43 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:25.964436 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42935b1f-03fc-47a2-9cda-8644434e2e43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:25.965605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42935b1f-03fc-47a2-9cda-8644434e2e43 None admin] RBAC: Adding query filter params (role.id=70ac549382b14b2cbb8b35e5dfc648d4, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=6362303cb918486c8525225914332ad5) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:25.966104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42935b1f-03fc-47a2-9cda-8644434e2e43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:25.967760 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42935b1f-03fc-47a2-9cda-8644434e2e43 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:25.973990 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 360/710] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:34:25 2022] GET /identity/v3/role_assignments?user.id=6362303cb918486c8525225914332ad5&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=70ac549382b14b2cbb8b35e5dfc648d4 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:25.975031 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42935b1f-03fc-47a2-9cda-8644434e2e43 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:25.997599 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dde01c-7edd-45bc-b693-e6718b8b8700 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:26.036167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d23b5-9ff9-4bf5-aade-6f533cb7f621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:26.037204 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d23b5-9ff9-4bf5-aade-6f533cb7f621 None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:26.037621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d23b5-9ff9-4bf5-aade-6f533cb7f621 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:26.038056 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d23b5-9ff9-4bf5-aade-6f533cb7f621 None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:26.038905 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d23b5-9ff9-4bf5-aade-6f533cb7f621 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:26.040554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d23b5-9ff9-4bf5-aade-6f533cb7f621 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:26.069408 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 351/711] 10.222.0.26 () {60 vars in 1132 bytes} [Mon May 9 09:34:25 2022] POST /identity/v3/users => generated 254 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:34:26.109078 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42935b1f-03fc-47a2-9cda-8644434e2e43 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:26.147933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc58fec-082a-42fe-9ac4-2f4fb7cf86c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:26.148920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc58fec-082a-42fe-9ac4-2f4fb7cf86c6 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:26.149193 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc58fec-082a-42fe-9ac4-2f4fb7cf86c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:26.149457 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc58fec-082a-42fe-9ac4-2f4fb7cf86c6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:26.150221 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc58fec-082a-42fe-9ac4-2f4fb7cf86c6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:26.151662 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc58fec-082a-42fe-9ac4-2f4fb7cf86c6 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:26.167355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-0bc58fec-082a-42fe-9ac4-2f4fb7cf86c6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 09:34:26.167355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:26.167355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:34:26.167355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:34:26.167355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:34:26.167355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:34:26.167355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:34:26.167355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:26.167355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 09:34:26.167355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:26.167355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:34:26.167355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:26.167355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:26.167355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:26.167355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:26.167355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:26.168411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:26.168411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:26.168411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:26.168411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:26.168411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:26.168411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:26.168411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:26.168411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:34:26.168411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:34:26.168411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:34:26.168411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:34:26.168411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:26.168411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:26.168411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:26.168411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:26.168411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:26.169339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:26.169339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:26.169339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:26.169339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:26.169339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:26.169339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:26.169339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:26.169339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:26.169339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:26.169339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:26.169339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:26.169339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:34:26.169339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:26.169339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 09:34:26.169339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:26.169339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 361/712] 10.222.0.26 () {56 vars in 1129 bytes} [Mon May 9 09:34:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:26.176994 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d23b5-9ff9-4bf5-aade-6f533cb7f621 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:26.294566 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d23b5-9ff9-4bf5-aade-6f533cb7f621 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:26.338518 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc5d02-7fec-42d0-9f63-b4e708b18c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:26.339195 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc5d02-7fec-42d0-9f63-b4e708b18c4d None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:26.339470 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc5d02-7fec-42d0-9f63-b4e708b18c4d None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:26.339737 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc5d02-7fec-42d0-9f63-b4e708b18c4d None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:26.340453 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc5d02-7fec-42d0-9f63-b4e708b18c4d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:26.340786 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc5d02-7fec-42d0-9f63-b4e708b18c4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:26.342230 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc5d02-7fec-42d0-9f63-b4e708b18c4d None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:26.347418 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 352/713] 10.222.0.26 () {56 vars in 1124 bytes} [Mon May 9 09:34:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:26.352932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc58fec-082a-42fe-9ac4-2f4fb7cf86c6 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:26.479869 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc58fec-082a-42fe-9ac4-2f4fb7cf86c6 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:26.515778 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad65283-538f-4a13-86a0-2a168a5d43d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:26.516446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad65283-538f-4a13-86a0-2a168a5d43d8 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:26.516755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad65283-538f-4a13-86a0-2a168a5d43d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:26.517002 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad65283-538f-4a13-86a0-2a168a5d43d8 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:26.517759 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad65283-538f-4a13-86a0-2a168a5d43d8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:26.518060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad65283-538f-4a13-86a0-2a168a5d43d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:26.519420 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad65283-538f-4a13-86a0-2a168a5d43d8 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:26.528387 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 362/714] 10.222.0.26 () {56 vars in 1124 bytes} [Mon May 9 09:34:26 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:26.673279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d8499-b5d0-4fdf-b191-ac02ee581fee None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:26.673666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d8499-b5d0-4fdf-b191-ac02ee581fee None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:26.673829 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d8499-b5d0-4fdf-b191-ac02ee581fee None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:26.675455 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 353/715] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:26.681992 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92d231-c7a5-4010-aef8-1d316b5fd509 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:26.682301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92d231-c7a5-4010-aef8-1d316b5fd509 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:26.682547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92d231-c7a5-4010-aef8-1d316b5fd509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:26.709857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-be92d231-c7a5-4010-aef8-1d316b5fd509 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:26.711074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-be92d231-c7a5-4010-aef8-1d316b5fd509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:26.796323 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 363/716] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:34:26.819065 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d8499-b5d0-4fdf-b191-ac02ee581fee None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:26.952890 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d8499-b5d0-4fdf-b191-ac02ee581fee None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:26.992587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f321035-f130-499f-9fef-8e282e8ef20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:26.993471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f321035-f130-499f-9fef-8e282e8ef20d None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:26.993783 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f321035-f130-499f-9fef-8e282e8ef20d None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:26.994048 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f321035-f130-499f-9fef-8e282e8ef20d None admin] PATH_INFO: `/v3/services/network` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:26.994712 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f321035-f130-499f-9fef-8e282e8ef20d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:26.996465 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f321035-f130-499f-9fef-8e282e8ef20d None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:27.016692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-1f321035-f130-499f-9fef-8e282e8ef20d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 09:34:27.016692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:27.016692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:34:27.016692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:34:27.016692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:34:27.016692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:34:27.016692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:34:27.016692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:27.016692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 09:34:27.016692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:27.016692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:34:27.016692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:27.016692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:27.016692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:27.016692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:27.016692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:27.016692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:27.017733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:27.017733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:27.017733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:27.017733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:27.017733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:27.017733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:27.017733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:34:27.017733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:34:27.017733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:34:27.017733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:34:27.017733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:27.017733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:27.017733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:27.017733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:27.017733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:27.017733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:27.018622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:27.018622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:27.018622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:27.018622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:27.018622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:27.018622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:27.018622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:27.018622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:27.018622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:27.018622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:27.018622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:34:27.018622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:27.018622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 09:34:27.018622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:27.018622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 354/717] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:26 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:27.024123 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92d231-c7a5-4010-aef8-1d316b5fd509 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:27.157039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92d231-c7a5-4010-aef8-1d316b5fd509 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:27.199600 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bfe7e2-52e9-4bbd-b957-64e97ab4b336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:27.200715 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfe7e2-52e9-4bbd-b957-64e97ab4b336 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:27.201192 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfe7e2-52e9-4bbd-b957-64e97ab4b336 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:27.201657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfe7e2-52e9-4bbd-b957-64e97ab4b336 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:27.202575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bfe7e2-52e9-4bbd-b957-64e97ab4b336 None admin] RBAC: Adding query filter params (name=network) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:27.203084 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bfe7e2-52e9-4bbd-b957-64e97ab4b336 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:27.204699 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bfe7e2-52e9-4bbd-b957-64e97ab4b336 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:27.212111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 364/718] 10.222.0.26 () {56 vars in 1112 bytes} [Mon May 9 09:34:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:27.217758 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f321035-f130-499f-9fef-8e282e8ef20d None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:27.354686 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f321035-f130-499f-9fef-8e282e8ef20d None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:27.391427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c4f86-e04d-4d15-b0a4-1469881a4e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:27.392107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c4f86-e04d-4d15-b0a4-1469881a4e91 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:27.392380 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c4f86-e04d-4d15-b0a4-1469881a4e91 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:27.392646 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c4f86-e04d-4d15-b0a4-1469881a4e91 None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:27.393441 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c4f86-e04d-4d15-b0a4-1469881a4e91 None admin] RBAC: Adding query filter params (type=network) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:27.393743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c4f86-e04d-4d15-b0a4-1469881a4e91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:27.395187 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c4f86-e04d-4d15-b0a4-1469881a4e91 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:27.404374 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 355/719] 10.222.0.26 () {56 vars in 1112 bytes} [Mon May 9 09:34:27 2022] GET /identity/v3/services?type=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:28.461197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935484a9-a058-4eb0-9e59-1c0ac4243e37 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:28.462454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935484a9-a058-4eb0-9e59-1c0ac4243e37 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:28.462454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935484a9-a058-4eb0-9e59-1c0ac4243e37 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:28.462454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 365/720] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:28.469875 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22ede6-abd6-47ce-9fe9-847855f8b53e None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:28.470449 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22ede6-abd6-47ce-9fe9-847855f8b53e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:28.470449 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22ede6-abd6-47ce-9fe9-847855f8b53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:28.493739 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-ec22ede6-abd6-47ce-9fe9-847855f8b53e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:28.494934 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-ec22ede6-abd6-47ce-9fe9-847855f8b53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:28.574044 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 356/721] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:28.608219 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935484a9-a058-4eb0-9e59-1c0ac4243e37 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:28.737131 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935484a9-a058-4eb0-9e59-1c0ac4243e37 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:28.770065 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acbdfb-4830-47e9-bb12-f87eeb350eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:28.771218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acbdfb-4830-47e9-bb12-f87eeb350eb5 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:28.771716 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acbdfb-4830-47e9-bb12-f87eeb350eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:28.772195 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acbdfb-4830-47e9-bb12-f87eeb350eb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:28.782092 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0acbdfb-4830-47e9-bb12-f87eeb350eb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:28.790006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-c0acbdfb-4830-47e9-bb12-f87eeb350eb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:34:28.790006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:28.790006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:28.790006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:28.790006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:28.790006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:28.790006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:28.790006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:28.790006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:28.790006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:28.790006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:28.790006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:28.790006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:28.790006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:28.790006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:28.790006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:28.790997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:28.790997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:28.790997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:28.790997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:28.790997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:28.790997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:28.790997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:28.790997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:28.790997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:28.790997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:28.790997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:28.790997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:28.790997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:28.790997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:28.790997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:28.790997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:28.791951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:28.791951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:28.791951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:28.791951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:28.791951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:28.791951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:28.791951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:28.791951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:28.791951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:28.791951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:28.791951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:34:28.791951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:28.794673 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 366/722] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:34:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:28.800267 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22ede6-abd6-47ce-9fe9-847855f8b53e None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:28.843305 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f46fa-84a3-46d4-8161-7370f68f19e5 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:28.843885 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f46fa-84a3-46d4-8161-7370f68f19e5 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:28.844328 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f46fa-84a3-46d4-8161-7370f68f19e5 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:28.846197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 367/723] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:34:28.851415 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c1150-4874-49c4-87e0-f912fde7047a None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:28.851949 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c1150-4874-49c4-87e0-f912fde7047a None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:28.852433 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c1150-4874-49c4-87e0-f912fde7047a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:28.868963 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-d61c1150-4874-49c4-87e0-f912fde7047a None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:28.870750 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-d61c1150-4874-49c4-87e0-f912fde7047a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:29.000761 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22ede6-abd6-47ce-9fe9-847855f8b53e None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:29.001159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 368/724] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 09:34:29.019498 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c1150-4874-49c4-87e0-f912fde7047a None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:29.061040 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce572c2e-afdc-409b-9775-5dbd2d1165d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:29.063445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce572c2e-afdc-409b-9775-5dbd2d1165d4 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:29.063943 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce572c2e-afdc-409b-9775-5dbd2d1165d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:29.064430 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce572c2e-afdc-409b-9775-5dbd2d1165d4 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:29.066349 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce572c2e-afdc-409b-9775-5dbd2d1165d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:29.067653 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce572c2e-afdc-409b-9775-5dbd2d1165d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:29.071077 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce572c2e-afdc-409b-9775-5dbd2d1165d4 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:29.095251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 357/725] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:34:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:29.100265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce572c2e-afdc-409b-9775-5dbd2d1165d4 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:29.150946 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c1150-4874-49c4-87e0-f912fde7047a None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:29.197281 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddafb93-1d92-4575-bc71-3c3c5b0f2526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:29.199053 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddafb93-1d92-4575-bc71-3c3c5b0f2526 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:29.199510 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddafb93-1d92-4575-bc71-3c3c5b0f2526 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:29.199970 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddafb93-1d92-4575-bc71-3c3c5b0f2526 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:29.215152 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddafb93-1d92-4575-bc71-3c3c5b0f2526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:29.222879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-9ddafb93-1d92-4575-bc71-3c3c5b0f2526 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:34:29.222879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:29.222879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:29.222879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:29.222879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:29.222879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:29.222879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:29.222879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:29.222879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:29.222879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:29.222879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:29.222879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:29.222879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:29.222879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:29.222879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:29.222879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:29.223832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:29.223832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:29.223832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:29.223832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:29.223832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:29.223832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:29.223832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:29.223832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:29.223832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:29.223832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:29.223832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:29.223832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:29.223832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:29.223832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:29.223832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:29.223832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:29.224704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:29.224704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:29.224704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:29.224704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:29.224704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:29.224704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:29.224704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:29.224704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:29.224704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:29.224704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:29.224704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:34:29.224704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:29.228230 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce572c2e-afdc-409b-9775-5dbd2d1165d4 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:29.229004 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 369/726] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:34:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:29.230358 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440d950-5021-4d84-9135-02e45db5dec1 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:29.230642 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440d950-5021-4d84-9135-02e45db5dec1 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:29.230915 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440d950-5021-4d84-9135-02e45db5dec1 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:29.232956 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 370/727] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:29.233204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440d950-5021-4d84-9135-02e45db5dec1 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:29.270746 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358d7e4-c852-4ab2-8920-f328d25ba412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:29.272037 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358d7e4-c852-4ab2-8920-f328d25ba412 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:29.273096 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358d7e4-c852-4ab2-8920-f328d25ba412 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:29.273376 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358d7e4-c852-4ab2-8920-f328d25ba412 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:29.290002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b358d7e4-c852-4ab2-8920-f328d25ba412 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:29.291793 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b358d7e4-c852-4ab2-8920-f328d25ba412 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:29.314640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-b358d7e4-c852-4ab2-8920-f328d25ba412 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:29.314640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:29.314640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:29.314640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:29.314640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:29.314640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:29.314640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:29.314640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:29.314640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:29.314640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:29.314640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:29.314640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:29.314640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:29.314640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:29.314640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:29.315587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:29.315587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:29.315587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:29.315587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:29.315587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:29.315587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:29.315587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:29.315587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:29.315587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:29.315587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:29.315587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:29.315587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:29.315587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:29.315587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:29.315587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:29.315587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:29.315587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:29.316504 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:29.316504 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:29.316504 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:29.316504 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:29.322734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 358/728] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:29.324010 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0124e-4c65-4b39-aeab-3ee888b6700d None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:29.324291 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0124e-4c65-4b39-aeab-3ee888b6700d None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:29.324563 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0124e-4c65-4b39-aeab-3ee888b6700d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:29.341135 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-d7e0124e-4c65-4b39-aeab-3ee888b6700d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:29.342327 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-d7e0124e-4c65-4b39-aeab-3ee888b6700d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:29.344758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440d950-5021-4d84-9135-02e45db5dec1 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:29.385291 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de48ef3-d878-4307-bb25-fa8f4f141c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:29.386406 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de48ef3-d878-4307-bb25-fa8f4f141c4e None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:29.386912 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de48ef3-d878-4307-bb25-fa8f4f141c4e None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:29.387280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de48ef3-d878-4307-bb25-fa8f4f141c4e None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:29.388141 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de48ef3-d878-4307-bb25-fa8f4f141c4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:29.388447 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de48ef3-d878-4307-bb25-fa8f4f141c4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:29.389859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de48ef3-d878-4307-bb25-fa8f4f141c4e None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:29.402197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 371/729] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:34:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:29.402197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de48ef3-d878-4307-bb25-fa8f4f141c4e None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:29.412597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 359/730] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:29.413794 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c17d1e-be30-47d3-ad75-2b90dd418a41 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:29.414071 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c17d1e-be30-47d3-ad75-2b90dd418a41 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:29.414337 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c17d1e-be30-47d3-ad75-2b90dd418a41 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:29.419843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 360/731] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:29.419843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c17d1e-be30-47d3-ad75-2b90dd418a41 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:29.504978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de48ef3-d878-4307-bb25-fa8f4f141c4e None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:29.513918 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c17d1e-be30-47d3-ad75-2b90dd418a41 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:29.536689 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bf0c9-0e49-4ada-b6d9-37820536d01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:29.537453 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bf0c9-0e49-4ada-b6d9-37820536d01a None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:29.537728 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bf0c9-0e49-4ada-b6d9-37820536d01a None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:29.538023 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bf0c9-0e49-4ada-b6d9-37820536d01a None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:29.538754 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bf0c9-0e49-4ada-b6d9-37820536d01a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:29.539054 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bf0c9-0e49-4ada-b6d9-37820536d01a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:29.540554 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bf0c9-0e49-4ada-b6d9-37820536d01a None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:29.549619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead50cc-6c24-48fe-ab64-56bd1c8e94a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:29.550419 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead50cc-6c24-48fe-ab64-56bd1c8e94a2 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:29.550689 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead50cc-6c24-48fe-ab64-56bd1c8e94a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:29.550960 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead50cc-6c24-48fe-ab64-56bd1c8e94a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:29.559361 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 372/732] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:29.559361 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd7b1f-8de8-46ec-97c3-f953aabf243c None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:29.559361 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd7b1f-8de8-46ec-97c3-f953aabf243c None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:29.559361 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd7b1f-8de8-46ec-97c3-f953aabf243c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:29.569714 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead50cc-6c24-48fe-ab64-56bd1c8e94a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:29.569954 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-58fd7b1f-8de8-46ec-97c3-f953aabf243c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:29.571085 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-58fd7b1f-8de8-46ec-97c3-f953aabf243c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:29.571186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead50cc-6c24-48fe-ab64-56bd1c8e94a2 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:29.588546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-bead50cc-6c24-48fe-ab64-56bd1c8e94a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:29.588546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:29.588546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:29.588546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:29.588546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:29.588546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:29.588546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:29.588546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:29.588546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:29.588546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:29.588546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:29.588546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:29.588546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:29.588546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:29.588546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:29.589634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:29.589634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:29.589634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:29.589634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:29.589634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:29.589634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:29.589634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:29.589634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:29.589634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:29.589634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:29.589634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:29.589634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:29.589634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:29.589634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:29.589634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:29.589634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:29.589634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:29.590527 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:29.590527 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:29.590527 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:29.590527 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:29.590527 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 361/733] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:29.591213 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead50cc-6c24-48fe-ab64-56bd1c8e94a2 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:29.635846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 373/734] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:29.637068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd7b1f-8de8-46ec-97c3-f953aabf243c None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:29.698886 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead50cc-6c24-48fe-ab64-56bd1c8e94a2 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:29.731476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd7b1f-8de8-46ec-97c3-f953aabf243c None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:29.731675 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796779a-0b7c-4874-9016-208bcc8b578a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:29.732405 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796779a-0b7c-4874-9016-208bcc8b578a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:29.732701 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796779a-0b7c-4874-9016-208bcc8b578a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:29.732974 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796779a-0b7c-4874-9016-208bcc8b578a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:29.743202 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796779a-0b7c-4874-9016-208bcc8b578a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:29.751217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-1796779a-0b7c-4874-9016-208bcc8b578a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:29.751217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:29.751217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:29.751217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:29.751217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:29.751217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:29.751217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:29.751217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:29.751217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:29.751217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:29.751217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:29.751217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:29.751217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:29.751217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:29.751217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:29.751217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:29.752173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:29.752173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:29.752173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:29.752173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:29.752173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:29.752173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:29.752173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:29.752173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:29.752173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:29.752173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:29.752173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:29.752173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:29.752173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:29.752173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:29.752173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:29.752173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:29.753047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:29.753047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:29.753047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:29.753047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:29.753047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:29.753047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:29.753047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:29.753047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:29.753047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:29.753047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:29.753047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:29.753047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:29.753047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 362/735] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:34:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:29.753753 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796779a-0b7c-4874-9016-208bcc8b578a None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:29.771018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49166b60-07e4-477c-bcc1-9ff675b9423a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:29.771818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49166b60-07e4-477c-bcc1-9ff675b9423a None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:29.772094 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49166b60-07e4-477c-bcc1-9ff675b9423a None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:29.772343 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49166b60-07e4-477c-bcc1-9ff675b9423a None admin] PATH_INFO: `/v3/users/swift` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:29.779790 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49166b60-07e4-477c-bcc1-9ff675b9423a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:29.781506 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49166b60-07e4-477c-bcc1-9ff675b9423a None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:29.796575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-49166b60-07e4-477c-bcc1-9ff675b9423a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 09:34:29.796575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:29.796575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:29.796575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:29.796575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:29.796575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:29.796575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:29.796575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:29.796575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:29.796575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:29.796575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:29.796575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:29.796575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:29.796575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:29.796575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:29.796575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:29.797799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:29.797799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:29.797799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:29.797799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:29.797799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:29.797799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:29.797799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:29.797799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:29.797799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:29.797799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:29.797799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:29.797799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:29.797799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:29.797799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:29.797799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:29.797799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:29.797799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:29.797799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:29.799018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:29.799018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:29.799018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:29.799018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:29.799018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:29.799018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:29.799018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:29.799018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:29.799018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 09:34:29.799018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:29.800253 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 374/736] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:34:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:29.801179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49166b60-07e4-477c-bcc1-9ff675b9423a None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:29.858791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796779a-0b7c-4874-9016-208bcc8b578a None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:29.892072 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e0614-ef50-40d4-835c-87da65a12fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:29.892747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e0614-ef50-40d4-835c-87da65a12fb9 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:29.893023 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e0614-ef50-40d4-835c-87da65a12fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:29.893295 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e0614-ef50-40d4-835c-87da65a12fb9 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:29.894003 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e0614-ef50-40d4-835c-87da65a12fb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:29.894300 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e0614-ef50-40d4-835c-87da65a12fb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:29.895851 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e0614-ef50-40d4-835c-87da65a12fb9 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:29.898106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49166b60-07e4-477c-bcc1-9ff675b9423a None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:29.907467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 363/737] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:29.908441 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e0614-ef50-40d4-835c-87da65a12fb9 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:29.938199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1097a-311d-40e9-9dfc-522ff09b925f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:29.938863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1097a-311d-40e9-9dfc-522ff09b925f None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:29.939187 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1097a-311d-40e9-9dfc-522ff09b925f None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:29.939395 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1097a-311d-40e9-9dfc-522ff09b925f None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:29.940111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1097a-311d-40e9-9dfc-522ff09b925f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:29.941532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1097a-311d-40e9-9dfc-522ff09b925f None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:29.968693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 375/738] 10.222.0.26 () {60 vars in 1141 bytes} [Mon May 9 09:34:29 2022] POST /identity/v3/services => generated 251 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:34:29.968693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1097a-311d-40e9-9dfc-522ff09b925f None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:30.036621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e0614-ef50-40d4-835c-87da65a12fb9 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:30.080200 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e91ef0-abe4-4d86-b547-c6b8029ae096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:30.080988 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e91ef0-abe4-4d86-b547-c6b8029ae096 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:30.081378 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e91ef0-abe4-4d86-b547-c6b8029ae096 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:30.081707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e91ef0-abe4-4d86-b547-c6b8029ae096 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:30.082546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e91ef0-abe4-4d86-b547-c6b8029ae096 None admin] RBAC: Adding query filter params (name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:30.082912 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e91ef0-abe4-4d86-b547-c6b8029ae096 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:30.084385 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e91ef0-abe4-4d86-b547-c6b8029ae096 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:30.093824 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 364/739] 10.222.0.26 () {56 vars in 1103 bytes} [Mon May 9 09:34:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:30.094782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e91ef0-abe4-4d86-b547-c6b8029ae096 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:30.100095 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1097a-311d-40e9-9dfc-522ff09b925f None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:30.144056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5612f6e-75df-4e18-809b-4b8711d24769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:30.144926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5612f6e-75df-4e18-809b-4b8711d24769 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:30.145208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5612f6e-75df-4e18-809b-4b8711d24769 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:30.145885 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5612f6e-75df-4e18-809b-4b8711d24769 None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:30.147006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5612f6e-75df-4e18-809b-4b8711d24769 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:30.147314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5612f6e-75df-4e18-809b-4b8711d24769 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:30.149130 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5612f6e-75df-4e18-809b-4b8711d24769 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:30.171196 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 376/740] 10.222.0.26 () {56 vars in 1135 bytes} [Mon May 9 09:34:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:30.172455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5612f6e-75df-4e18-809b-4b8711d24769 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:30.208777 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e91ef0-abe4-4d86-b547-c6b8029ae096 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:30.243442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12edb927-3496-4d0a-9953-af18a1336671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:30.244233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edb927-3496-4d0a-9953-af18a1336671 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:30.244506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edb927-3496-4d0a-9953-af18a1336671 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:30.244807 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edb927-3496-4d0a-9953-af18a1336671 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:30.252638 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12edb927-3496-4d0a-9953-af18a1336671 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:30.254108 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12edb927-3496-4d0a-9953-af18a1336671 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:30.268263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-12edb927-3496-4d0a-9953-af18a1336671 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 09:34:30.268263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:30.268263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:30.268263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:30.268263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:30.268263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:30.268263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:30.268263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:30.268263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:30.268263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:30.268263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:30.268263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:30.268263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:30.268263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:30.268263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:30.268263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:30.269429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:30.269429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:30.269429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:30.269429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:30.269429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:30.269429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:30.269429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:30.269429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:30.269429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:30.269429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:30.269429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:30.269429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:30.269429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:30.269429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:30.269429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:30.269429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:30.269429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:30.269429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:30.270445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:30.270445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:30.270445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:30.270445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:30.270445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:30.270445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:30.270445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:30.270445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:30.270445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 09:34:30.270445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:30.270445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 365/741] 10.222.0.26 () {56 vars in 1093 bytes} [Mon May 9 09:34:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:30.271071 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12edb927-3496-4d0a-9953-af18a1336671 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:30.271791 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5612f6e-75df-4e18-809b-4b8711d24769 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:30.305300 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ea777-09a3-42c4-ae34-a8cfc3474581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:30.306127 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ea777-09a3-42c4-ae34-a8cfc3474581 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:30.306402 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ea777-09a3-42c4-ae34-a8cfc3474581 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:30.306671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ea777-09a3-42c4-ae34-a8cfc3474581 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:30.316997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ea777-09a3-42c4-ae34-a8cfc3474581 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:30.318447 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ea777-09a3-42c4-ae34-a8cfc3474581 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:30.335534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-619ea777-09a3-42c4-ae34-a8cfc3474581 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:30.335534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:30.335534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:30.335534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:30.335534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:30.335534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:30.335534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:30.335534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:30.335534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:30.335534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:30.335534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:30.335534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:30.335534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:30.335534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:30.335534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:30.336608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:30.336608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:30.336608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:30.336608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:30.336608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:30.336608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:30.336608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:30.336608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:30.336608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:30.336608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:30.336608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:30.336608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:30.336608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:30.336608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:30.336608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:30.336608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:30.336608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:30.337851 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:30.337851 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:30.337851 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:30.337851 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:30.340861 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 377/742] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:30.342383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0538739-d71d-4225-b858-be855d639dd8 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:30.342804 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0538739-d71d-4225-b858-be855d639dd8 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:30.343207 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0538739-d71d-4225-b858-be855d639dd8 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:30.344802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 378/743] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:30.345611 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0538739-d71d-4225-b858-be855d639dd8 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:30.377800 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12edb927-3496-4d0a-9953-af18a1336671 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:30.410627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94f392-c600-45ef-b294-1fd7fde23e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:30.411425 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f392-c600-45ef-b294-1fd7fde23e7e None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:30.411702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f392-c600-45ef-b294-1fd7fde23e7e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:30.411965 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f392-c600-45ef-b294-1fd7fde23e7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:30.444027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0538739-d71d-4225-b858-be855d639dd8 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:30.475617 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447cfc7c-8d84-43ce-81fe-d6a3b1e9c470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:30.476308 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cfc7c-8d84-43ce-81fe-d6a3b1e9c470 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:30.476556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cfc7c-8d84-43ce-81fe-d6a3b1e9c470 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:30.476878 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cfc7c-8d84-43ce-81fe-d6a3b1e9c470 None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:30.477773 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447cfc7c-8d84-43ce-81fe-d6a3b1e9c470 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:30.478075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447cfc7c-8d84-43ce-81fe-d6a3b1e9c470 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:30.479498 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447cfc7c-8d84-43ce-81fe-d6a3b1e9c470 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:30.484008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94f392-c600-45ef-b294-1fd7fde23e7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:30.485517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94f392-c600-45ef-b294-1fd7fde23e7e None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:30.497476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 379/744] 10.222.0.26 () {56 vars in 1133 bytes} [Mon May 9 09:34:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:30.498473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447cfc7c-8d84-43ce-81fe-d6a3b1e9c470 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:30.584522 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 366/745] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:34:30 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 09:34:30.586643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7621b9-f5bd-4a83-8d9f-839996eb1f2f None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:30.587209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7621b9-f5bd-4a83-8d9f-839996eb1f2f None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:30.587677 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7621b9-f5bd-4a83-8d9f-839996eb1f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:30.592748 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447cfc7c-8d84-43ce-81fe-d6a3b1e9c470 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:30.603822 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-ad7621b9-f5bd-4a83-8d9f-839996eb1f2f None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:30.605025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-ad7621b9-f5bd-4a83-8d9f-839996eb1f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:30.626155 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148efe7f-1367-42c1-bc94-a0944228220c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:30.626815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148efe7f-1367-42c1-bc94-a0944228220c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:30.627092 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148efe7f-1367-42c1-bc94-a0944228220c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:30.627362 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148efe7f-1367-42c1-bc94-a0944228220c None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:30.628078 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148efe7f-1367-42c1-bc94-a0944228220c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:30.628378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148efe7f-1367-42c1-bc94-a0944228220c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:30.629767 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148efe7f-1367-42c1-bc94-a0944228220c None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:30.640559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 380/746] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:30.641368 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148efe7f-1367-42c1-bc94-a0944228220c None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:30.672819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 367/747] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 09:34:30.672819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7621b9-f5bd-4a83-8d9f-839996eb1f2f None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:30.740147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148efe7f-1367-42c1-bc94-a0944228220c None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:30.772906 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7621b9-f5bd-4a83-8d9f-839996eb1f2f None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:30.774489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf17fb-b3f6-4186-ab67-15a3b08f3add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:30.775378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf17fb-b3f6-4186-ab67-15a3b08f3add None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:30.775638 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf17fb-b3f6-4186-ab67-15a3b08f3add None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:30.775926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf17fb-b3f6-4186-ab67-15a3b08f3add None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:30.786561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebf17fb-b3f6-4186-ab67-15a3b08f3add None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:30.788313 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebf17fb-b3f6-4186-ab67-15a3b08f3add None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:30.805332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-8ebf17fb-b3f6-4186-ab67-15a3b08f3add None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:30.805332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:30.805332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:30.805332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:30.805332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:30.805332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:30.805332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:30.805332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:30.805332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:30.805332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:30.805332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:30.805332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:30.805332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:30.805332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:30.805332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:30.806320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:30.806320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:30.806320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:30.806320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:30.806320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:30.806320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:30.806320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:30.806320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:30.806320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:30.806320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:30.806320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:30.806320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:30.806320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:30.806320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:30.806320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:30.806320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:30.806320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:30.807237 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:30.807237 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:30.807237 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:30.807237 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:30.810039 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9125ce-630c-41ee-957d-f26542a59e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:30.810868 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9125ce-630c-41ee-957d-f26542a59e96 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:30.811136 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9125ce-630c-41ee-957d-f26542a59e96 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:30.811401 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9125ce-630c-41ee-957d-f26542a59e96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:30.812132 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 381/748] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:30.813560 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf17fb-b3f6-4186-ab67-15a3b08f3add None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:30.822338 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9125ce-630c-41ee-957d-f26542a59e96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:30.823806 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9125ce-630c-41ee-957d-f26542a59e96 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:30.841351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-7e9125ce-630c-41ee-957d-f26542a59e96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:30.841351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:30.841351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:30.841351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:30.841351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:30.841351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:30.841351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:30.841351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:30.841351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:30.841351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:30.841351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:30.841351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:30.841351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:30.841351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:30.841351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:30.842281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:30.842281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:30.842281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:30.842281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:30.842281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:30.842281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:30.842281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:30.842281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:30.842281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:30.842281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:30.842281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:30.842281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:30.842281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:30.842281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:30.842281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:30.842281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:30.842281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:30.843169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:30.843169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:30.843169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:30.843169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:30.843169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 368/749] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:30.843456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9125ce-630c-41ee-957d-f26542a59e96 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:30.916747 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf17fb-b3f6-4186-ab67-15a3b08f3add None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:30.947588 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9125ce-630c-41ee-957d-f26542a59e96 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:30.954706 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73e843-b9c8-4edc-85ee-75f8a4ba5f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:30.955532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73e843-b9c8-4edc-85ee-75f8a4ba5f59 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:30.955812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73e843-b9c8-4edc-85ee-75f8a4ba5f59 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:30.956083 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73e843-b9c8-4edc-85ee-75f8a4ba5f59 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:30.964600 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73e843-b9c8-4edc-85ee-75f8a4ba5f59 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:30.966100 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73e843-b9c8-4edc-85ee-75f8a4ba5f59 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:30.981890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-ca73e843-b9c8-4edc-85ee-75f8a4ba5f59 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 09:34:30.981890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:30.981890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:30.981890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:30.981890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:30.981890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:30.981890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:30.981890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:30.981890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:30.981890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:30.981890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:30.981890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:30.981890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:30.981890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:30.981890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:30.981890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:30.982922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:30.982922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:30.982922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:30.982922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:30.982922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:30.982922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:30.982922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:30.982922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:30.982922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:30.982922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:30.982922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:30.982922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:30.982922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:30.982922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:30.982922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:30.982922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:30.982922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:30.982922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:30.984516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:30.984516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:30.984516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:30.984516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:30.984516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:30.984516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:30.984516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:30.984516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:30.984516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 09:34:30.984516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:30.984516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 382/750] 10.222.0.26 () {56 vars in 1117 bytes} [Mon May 9 09:34:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:30.984516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73e843-b9c8-4edc-85ee-75f8a4ba5f59 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:31.094155 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73e843-b9c8-4edc-85ee-75f8a4ba5f59 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:31.114407 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b8503-2541-4c38-b6e6-e076d89d9c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:31.115072 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b8503-2541-4c38-b6e6-e076d89d9c5e None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:31.115347 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b8503-2541-4c38-b6e6-e076d89d9c5e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:31.115617 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b8503-2541-4c38-b6e6-e076d89d9c5e None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:31.116344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b8503-2541-4c38-b6e6-e076d89d9c5e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:31.117758 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b8503-2541-4c38-b6e6-e076d89d9c5e None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:31.129474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ce8a8-001a-42f2-aa55-8a782ce1ab1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:31.130139 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ce8a8-001a-42f2-aa55-8a782ce1ab1b None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:31.130454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ce8a8-001a-42f2-aa55-8a782ce1ab1b None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:31.130681 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ce8a8-001a-42f2-aa55-8a782ce1ab1b None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:31.131418 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ce8a8-001a-42f2-aa55-8a782ce1ab1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:31.131736 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ce8a8-001a-42f2-aa55-8a782ce1ab1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:31.133110 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ce8a8-001a-42f2-aa55-8a782ce1ab1b None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:31.133649 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 369/751] 10.222.0.26 () {60 vars in 1141 bytes} [Mon May 9 09:34:30 2022] POST /identity/v3/services => generated 240 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:34:31.134722 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b8503-2541-4c38-b6e6-e076d89d9c5e None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:31.164867 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 383/752] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:31.166149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ce8a8-001a-42f2-aa55-8a782ce1ab1b None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:31.285880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b8503-2541-4c38-b6e6-e076d89d9c5e None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:31.315174 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ce8a8-001a-42f2-aa55-8a782ce1ab1b None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:31.337250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9b53a-ec59-42af-b949-424a9159fc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:31.337927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9b53a-ec59-42af-b949-424a9159fc6f None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:31.338197 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9b53a-ec59-42af-b949-424a9159fc6f None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:31.338463 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9b53a-ec59-42af-b949-424a9159fc6f None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:31.339183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c9b53a-ec59-42af-b949-424a9159fc6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:31.339481 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c9b53a-ec59-42af-b949-424a9159fc6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:31.341109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c9b53a-ec59-42af-b949-424a9159fc6f None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:31.353635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 370/753] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:31.354396 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9b53a-ec59-42af-b949-424a9159fc6f None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:31.357204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc6636-4817-494d-9774-6c8c6c0265ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:31.358123 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc6636-4817-494d-9774-6c8c6c0265ae None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:31.358408 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc6636-4817-494d-9774-6c8c6c0265ae None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:31.358682 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc6636-4817-494d-9774-6c8c6c0265ae None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:31.359529 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc6636-4817-494d-9774-6c8c6c0265ae None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:31.359821 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc6636-4817-494d-9774-6c8c6c0265ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:31.361296 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc6636-4817-494d-9774-6c8c6c0265ae None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:31.384220 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 384/754] 10.222.0.26 () {56 vars in 1149 bytes} [Mon May 9 09:34:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:31.385167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc6636-4817-494d-9774-6c8c6c0265ae None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:31.479310 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9b53a-ec59-42af-b949-424a9159fc6f None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:31.488566 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc6636-4817-494d-9774-6c8c6c0265ae None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:31.515036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b98c3-eef6-48d7-bef4-815d0c73b928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:31.515904 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b98c3-eef6-48d7-bef4-815d0c73b928 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:31.516176 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b98c3-eef6-48d7-bef4-815d0c73b928 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:31.516447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b98c3-eef6-48d7-bef4-815d0c73b928 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:31.523532 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b98c3-eef6-48d7-bef4-815d0c73b928 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:31.525125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b98c3-eef6-48d7-bef4-815d0c73b928 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:31.526965 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee1506-d82e-4101-8bd8-50d949f33550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:31.527930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee1506-d82e-4101-8bd8-50d949f33550 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:31.528344 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee1506-d82e-4101-8bd8-50d949f33550 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:31.528808 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee1506-d82e-4101-8bd8-50d949f33550 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:31.535617 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee1506-d82e-4101-8bd8-50d949f33550 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:31.537235 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee1506-d82e-4101-8bd8-50d949f33550 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:31.539350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-207b98c3-eef6-48d7-bef4-815d0c73b928 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:31.539350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:31.539350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:31.539350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:31.539350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:31.539350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:31.539350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:31.539350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:31.539350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:31.539350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:31.539350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:31.539350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:31.539350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:31.539350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:31.539350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:31.540436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:31.540436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:31.540436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:31.540436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:31.540436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:31.540436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:31.540436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:31.540436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:31.540436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:31.540436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:31.540436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:31.540436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:31.540436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:31.540436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:31.540436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:31.540436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:31.540436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:31.541513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:31.541513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:31.541513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:31.541513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:31.541513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:31.541513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:31.541513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:31.541513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:31.545026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 371/755] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:31.545935 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b98c3-eef6-48d7-bef4-815d0c73b928 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:31.554357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-0fee1506-d82e-4101-8bd8-50d949f33550 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:31.554357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:31.554357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:31.554357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:31.554357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:31.554357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:31.554357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:31.554357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:31.554357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:31.554357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:31.554357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:31.554357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:31.554357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:31.554357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:31.554357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:31.555455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:31.555455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:31.555455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:31.555455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:31.555455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:31.555455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:31.555455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:31.555455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:31.555455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:31.555455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:31.555455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:31.555455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:31.555455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:31.555455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:31.555455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:31.555455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:31.555455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:31.556605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:31.556605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:31.556605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:31.556605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:31.556605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:31.556605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:31.556605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:31.556605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:31.560178 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 385/756] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:31.561265 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee1506-d82e-4101-8bd8-50d949f33550 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:31.653907 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b98c3-eef6-48d7-bef4-815d0c73b928 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:31.662012 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee1506-d82e-4101-8bd8-50d949f33550 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:31.689366 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd1756-ffcd-47f8-b41a-9dc56976d8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:31.690025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd1756-ffcd-47f8-b41a-9dc56976d8ee None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:31.690321 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd1756-ffcd-47f8-b41a-9dc56976d8ee None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:31.690575 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd1756-ffcd-47f8-b41a-9dc56976d8ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:31.698981 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e611c3-a43f-4512-9cff-9b4c4bdff847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:31.699929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e611c3-a43f-4512-9cff-9b4c4bdff847 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:31.700344 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e611c3-a43f-4512-9cff-9b4c4bdff847 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:31.700782 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e611c3-a43f-4512-9cff-9b4c4bdff847 None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:31.701708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e611c3-a43f-4512-9cff-9b4c4bdff847 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:31.702151 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e611c3-a43f-4512-9cff-9b4c4bdff847 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:31.703734 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e611c3-a43f-4512-9cff-9b4c4bdff847 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:31.717434 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 386/757] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:31.718559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e611c3-a43f-4512-9cff-9b4c4bdff847 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:31.771163 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd1756-ffcd-47f8-b41a-9dc56976d8ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:31.773939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd1756-ffcd-47f8-b41a-9dc56976d8ee None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:31.828510 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e611c3-a43f-4512-9cff-9b4c4bdff847 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:31.870840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2be91e-9de1-4335-93cf-46011d3e54c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:31.871682 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2be91e-9de1-4335-93cf-46011d3e54c1 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:31.871965 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2be91e-9de1-4335-93cf-46011d3e54c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:31.872263 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2be91e-9de1-4335-93cf-46011d3e54c1 None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:31.873692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2be91e-9de1-4335-93cf-46011d3e54c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:31.874006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2be91e-9de1-4335-93cf-46011d3e54c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:31.875915 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2be91e-9de1-4335-93cf-46011d3e54c1 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:31.882079 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 372/758] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:34:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 09:34:31.883062 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd1756-ffcd-47f8-b41a-9dc56976d8ee None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:31.899939 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 387/759] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:31.901577 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2be91e-9de1-4335-93cf-46011d3e54c1 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:31.988706 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd1756-ffcd-47f8-b41a-9dc56976d8ee None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:32.011110 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2be91e-9de1-4335-93cf-46011d3e54c1 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:32.022316 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42547f-5d3f-41a2-b613-c86bff15a427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:32.023198 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42547f-5d3f-41a2-b613-c86bff15a427 None admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:32.023467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42547f-5d3f-41a2-b613-c86bff15a427 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:32.023736 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42547f-5d3f-41a2-b613-c86bff15a427 None admin] PATH_INFO: `/v3/projects/9cd39464055141b8b43fde20d0885c99/users/6362303cb918486c8525225914332ad5/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:32.025828 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42547f-5d3f-41a2-b613-c86bff15a427 None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd39464055141b8b43fde20d0885c99, user_id=6362303cb918486c8525225914332ad5, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:32.027375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42547f-5d3f-41a2-b613-c86bff15a427 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:32.048950 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 373/760] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:34:31 2022] PUT /identity/v3/projects/9cd39464055141b8b43fde20d0885c99/users/6362303cb918486c8525225914332ad5/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:34:32.048950 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42547f-5d3f-41a2-b613-c86bff15a427 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:32.074567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545f1be-216b-4a95-9321-76ba342784cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:32.075438 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545f1be-216b-4a95-9321-76ba342784cb None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:32.075720 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545f1be-216b-4a95-9321-76ba342784cb None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:32.076036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545f1be-216b-4a95-9321-76ba342784cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:32.100642 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545f1be-216b-4a95-9321-76ba342784cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:32.106030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545f1be-216b-4a95-9321-76ba342784cb None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:32.148947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-a545f1be-216b-4a95-9321-76ba342784cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:32.148947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:32.148947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:32.148947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:32.148947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:32.148947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:32.148947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:32.148947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:32.148947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:32.148947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:32.148947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:32.148947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:32.148947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:32.148947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:32.148947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:32.150068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:32.150068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:32.150068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:32.150068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:32.150068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:32.150068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:32.150068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:32.150068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:32.150068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:32.150068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:32.150068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:32.150068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:32.150068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:32.150068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:32.150068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:32.150068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:32.150068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:32.150999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:32.150999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:32.150999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:32.150999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:32.156841 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 388/761] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:32.157802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545f1be-216b-4a95-9321-76ba342784cb None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:32.208846 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42547f-5d3f-41a2-b613-c86bff15a427 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:32.251423 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22958404-c4dd-4d3b-a38e-cc5b230744db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:32.252008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22958404-c4dd-4d3b-a38e-cc5b230744db None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:32.252279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22958404-c4dd-4d3b-a38e-cc5b230744db None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:32.252544 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22958404-c4dd-4d3b-a38e-cc5b230744db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:32.253478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22958404-c4dd-4d3b-a38e-cc5b230744db None admin] RBAC: Adding query filter params (role.id=70ac549382b14b2cbb8b35e5dfc648d4, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=380e04358f2349aa912d6a97518877cd) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:32.253779 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22958404-c4dd-4d3b-a38e-cc5b230744db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:32.255283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22958404-c4dd-4d3b-a38e-cc5b230744db None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:32.268417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 374/762] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:34:32 2022] GET /identity/v3/role_assignments?user.id=380e04358f2349aa912d6a97518877cd&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=70ac549382b14b2cbb8b35e5dfc648d4 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:32.311020 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545f1be-216b-4a95-9321-76ba342784cb None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:32.333962 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a81e6-618c-46cd-8786-d9a17c631acf None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:32.334200 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a81e6-618c-46cd-8786-d9a17c631acf None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:32.334293 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a81e6-618c-46cd-8786-d9a17c631acf None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:32.335951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 375/763] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:34:32.341927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f99f2b-215e-454b-964c-a0b09c1e1266 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:32.342254 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f99f2b-215e-454b-964c-a0b09c1e1266 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:32.342486 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f99f2b-215e-454b-964c-a0b09c1e1266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:32.359403 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc52f7-67cd-48e0-a9b0-9fa603195028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:32.360097 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc52f7-67cd-48e0-a9b0-9fa603195028 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:32.360376 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc52f7-67cd-48e0-a9b0-9fa603195028 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:32.360647 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc52f7-67cd-48e0-a9b0-9fa603195028 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:32.361416 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc52f7-67cd-48e0-a9b0-9fa603195028 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:32.361719 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc52f7-67cd-48e0-a9b0-9fa603195028 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:32.363118 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc52f7-67cd-48e0-a9b0-9fa603195028 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:32.372402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-a6f99f2b-215e-454b-964c-a0b09c1e1266 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:32.374449 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a6f99f2b-215e-454b-964c-a0b09c1e1266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:32.380720 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 389/764] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:32.388797 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc52f7-67cd-48e0-a9b0-9fa603195028 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:32.495692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 376/765] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 09:34:32.496010 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc52f7-67cd-48e0-a9b0-9fa603195028 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:32.532847 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f99f2b-215e-454b-964c-a0b09c1e1266 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:32.540957 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e77c93-3ff6-4e6e-8620-03dbb50f34f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:32.542186 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e77c93-3ff6-4e6e-8620-03dbb50f34f4 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:32.542609 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e77c93-3ff6-4e6e-8620-03dbb50f34f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:32.543012 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e77c93-3ff6-4e6e-8620-03dbb50f34f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:32.555848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e77c93-3ff6-4e6e-8620-03dbb50f34f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:32.557536 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e77c93-3ff6-4e6e-8620-03dbb50f34f4 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:32.587806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-f2e77c93-3ff6-4e6e-8620-03dbb50f34f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:32.587806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:32.587806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:32.587806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:32.587806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:32.587806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:32.587806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:32.587806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:32.587806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:32.587806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:32.587806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:32.587806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:32.587806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:32.587806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:32.587806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:32.589007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:32.589007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:32.589007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:32.589007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:32.589007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:32.589007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:32.589007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:32.589007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:32.589007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:32.589007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:32.589007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:32.589007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:32.589007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:32.589007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:32.589007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:32.589007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:32.589007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:32.590142 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:32.590142 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:32.590142 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:32.590142 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:32.590142 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:32.590142 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:32.590142 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:32.590142 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:32.601039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 390/766] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:32.602146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e77c93-3ff6-4e6e-8620-03dbb50f34f4 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:32.629287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f99f2b-215e-454b-964c-a0b09c1e1266 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:32.665620 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb0a5a7-3020-431e-b945-926b518561ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:32.666658 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0a5a7-3020-431e-b945-926b518561ac None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:32.666796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0a5a7-3020-431e-b945-926b518561ac None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:32.666944 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0a5a7-3020-431e-b945-926b518561ac None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:32.670017 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb0a5a7-3020-431e-b945-926b518561ac None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:32.670017 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb0a5a7-3020-431e-b945-926b518561ac None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:32.692083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-beb0a5a7-3020-431e-b945-926b518561ac None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 09:34:32.692083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:32.692083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:34:32.692083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:34:32.692083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:34:32.692083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:34:32.692083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:34:32.692083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:32.692083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 09:34:32.692083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:32.692083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:34:32.692083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:32.692083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:32.692083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:32.692083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:32.692083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:32.693160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:32.693160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:32.693160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:32.693160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:32.693160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:32.693160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:32.693160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:32.693160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:34:32.693160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:34:32.693160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:34:32.693160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:34:32.693160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:32.693160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:32.693160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:32.693160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:32.693160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:32.694036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:32.694036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:32.694036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:32.694036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:32.694036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:32.694036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:32.694036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:32.694036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:32.694036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:32.694036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:32.694036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:32.694036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:34:32.694036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:32.694036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 09:34:32.694036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:32.694036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 377/767] 10.222.0.26 () {56 vars in 1129 bytes} [Mon May 9 09:34:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:32.702245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb0a5a7-3020-431e-b945-926b518561ac None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:32.758459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e77c93-3ff6-4e6e-8620-03dbb50f34f4 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:32.796758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57041f73-f3bd-4f7f-a449-845d588705c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:32.797806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57041f73-f3bd-4f7f-a449-845d588705c2 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:32.798284 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57041f73-f3bd-4f7f-a449-845d588705c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:32.798818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57041f73-f3bd-4f7f-a449-845d588705c2 None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:32.799825 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57041f73-f3bd-4f7f-a449-845d588705c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:32.802056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57041f73-f3bd-4f7f-a449-845d588705c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:32.803707 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57041f73-f3bd-4f7f-a449-845d588705c2 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:32.820027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 391/768] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:32.826563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57041f73-f3bd-4f7f-a449-845d588705c2 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:32.938840 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb0a5a7-3020-431e-b945-926b518561ac None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:32.943588 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57041f73-f3bd-4f7f-a449-845d588705c2 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:32.986311 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861eee6f-369a-42ae-b321-f2e8ecabecc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:32.987370 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861eee6f-369a-42ae-b321-f2e8ecabecc7 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:32.987877 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861eee6f-369a-42ae-b321-f2e8ecabecc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:32.988418 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861eee6f-369a-42ae-b321-f2e8ecabecc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:32.989595 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861eee6f-369a-42ae-b321-f2e8ecabecc7 None admin] RBAC: Adding query filter params (role.id=f55c3728df78466989d0cd1b13c5ea28, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=af62b2d3f73f482b8db4173af33752c6) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:32.990131 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861eee6f-369a-42ae-b321-f2e8ecabecc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:32.993185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861eee6f-369a-42ae-b321-f2e8ecabecc7 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:33.005252 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 392/769] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:34:32 2022] GET /identity/v3/role_assignments?user.id=af62b2d3f73f482b8db4173af33752c6&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=f55c3728df78466989d0cd1b13c5ea28 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:33.022774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220cb49-4fca-4a26-9932-de32010cfb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:33.023861 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220cb49-4fca-4a26-9932-de32010cfb33 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:33.024393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220cb49-4fca-4a26-9932-de32010cfb33 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:33.025296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220cb49-4fca-4a26-9932-de32010cfb33 None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:33.026251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220cb49-4fca-4a26-9932-de32010cfb33 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:33.026756 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220cb49-4fca-4a26-9932-de32010cfb33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:33.028408 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220cb49-4fca-4a26-9932-de32010cfb33 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:33.034286 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 378/770] 10.222.0.26 () {56 vars in 1124 bytes} [Mon May 9 09:34:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:33.041222 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861eee6f-369a-42ae-b321-f2e8ecabecc7 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:33.150444 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861eee6f-369a-42ae-b321-f2e8ecabecc7 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:33.185763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a1186-f0ad-4471-a35e-2a288de4fbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:33.186806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a1186-f0ad-4471-a35e-2a288de4fbf2 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:33.187286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a1186-f0ad-4471-a35e-2a288de4fbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:33.187755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a1186-f0ad-4471-a35e-2a288de4fbf2 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:33.188745 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a1186-f0ad-4471-a35e-2a288de4fbf2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:33.189247 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a1186-f0ad-4471-a35e-2a288de4fbf2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:33.190921 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a1186-f0ad-4471-a35e-2a288de4fbf2 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:33.197087 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 393/771] 10.222.0.26 () {56 vars in 1124 bytes} [Mon May 9 09:34:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:33.203257 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220cb49-4fca-4a26-9932-de32010cfb33 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:33.308704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220cb49-4fca-4a26-9932-de32010cfb33 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:33.342394 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec43d30-f239-4c9a-a71b-c8c4342b3e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:33.343402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec43d30-f239-4c9a-a71b-c8c4342b3e81 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:33.343878 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec43d30-f239-4c9a-a71b-c8c4342b3e81 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:33.344363 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec43d30-f239-4c9a-a71b-c8c4342b3e81 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:33.345397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec43d30-f239-4c9a-a71b-c8c4342b3e81 None admin] RBAC: Adding query filter params (interface=public, service_id=3118f250321443c58cc1458c1ec938f0, region_id=RegionOne) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:33.345901 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec43d30-f239-4c9a-a71b-c8c4342b3e81 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:33.347451 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec43d30-f239-4c9a-a71b-c8c4342b3e81 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:33.355618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 379/772] 10.222.0.26 () {56 vars in 1251 bytes} [Mon May 9 09:34:33 2022] GET /identity/v3/endpoints?service_id=3118f250321443c58cc1458c1ec938f0&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:33.361303 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a1186-f0ad-4471-a35e-2a288de4fbf2 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:33.459425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a1186-f0ad-4471-a35e-2a288de4fbf2 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:33.498041 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4ded1-9874-4b42-8809-ac6dc590145c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:33.499126 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4ded1-9874-4b42-8809-ac6dc590145c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:33.499605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4ded1-9874-4b42-8809-ac6dc590145c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:33.500100 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4ded1-9874-4b42-8809-ac6dc590145c None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:33.501044 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4ded1-9874-4b42-8809-ac6dc590145c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:33.502660 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4ded1-9874-4b42-8809-ac6dc590145c None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:33.508938 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 394/773] 10.222.0.26 () {56 vars in 1087 bytes} [Mon May 9 09:34:33 2022] GET /identity/v3/services => generated 772 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:33.672654 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797adf10-3723-49e2-a2eb-b963343c5dd7 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:33.673720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797adf10-3723-49e2-a2eb-b963343c5dd7 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:33.674207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797adf10-3723-49e2-a2eb-b963343c5dd7 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:33.675880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 380/774] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:33.682254 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ab13f-a4df-4085-8b29-547d381d4ea0 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:33.682847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ab13f-a4df-4085-8b29-547d381d4ea0 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:33.683358 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ab13f-a4df-4085-8b29-547d381d4ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:33.705736 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-6c9ab13f-a4df-4085-8b29-547d381d4ea0 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:33.707330 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-6c9ab13f-a4df-4085-8b29-547d381d4ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:33.778297 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 395/775] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:34:33.807816 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797adf10-3723-49e2-a2eb-b963343c5dd7 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:33.910350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797adf10-3723-49e2-a2eb-b963343c5dd7 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:33.943732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daae560-4cfc-4808-987e-112718383ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:33.944824 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daae560-4cfc-4808-987e-112718383ba1 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:33.945268 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daae560-4cfc-4808-987e-112718383ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:33.945678 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daae560-4cfc-4808-987e-112718383ba1 None admin] PATH_INFO: `/v3/services/network` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:33.946633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daae560-4cfc-4808-987e-112718383ba1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:33.948180 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daae560-4cfc-4808-987e-112718383ba1 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:33.960421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-7daae560-4cfc-4808-987e-112718383ba1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 09:34:33.960421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:33.960421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:34:33.960421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:34:33.960421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:34:33.960421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:34:33.960421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:34:33.960421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:33.960421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 09:34:33.960421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:33.960421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:34:33.960421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:33.960421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:33.960421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:33.960421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:33.960421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:33.960421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:33.961672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:33.961672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:33.961672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:33.961672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:33.961672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:33.961672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:33.961672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:34:33.961672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:34:33.961672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:34:33.961672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:34:33.961672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:33.961672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:33.961672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:33.961672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:33.961672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:33.961672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:33.962747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:33.962747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:33.962747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:33.962747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:33.962747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:33.962747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:33.962747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:33.962747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:33.962747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:33.962747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:33.962747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:34:33.962747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:33.962747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 09:34:33.962747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:33.965407 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 381/776] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:33 2022] GET /identity/v3/services/network => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:33.973538 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ab13f-a4df-4085-8b29-547d381d4ea0 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:34.104328 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ab13f-a4df-4085-8b29-547d381d4ea0 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:34.153396 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5245c46-a520-45f9-b154-c0cb33d5eff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:34.154084 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5245c46-a520-45f9-b154-c0cb33d5eff7 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:34.154365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5245c46-a520-45f9-b154-c0cb33d5eff7 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:34.154642 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5245c46-a520-45f9-b154-c0cb33d5eff7 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:34.155358 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5245c46-a520-45f9-b154-c0cb33d5eff7 None admin] RBAC: Adding query filter params (name=network) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:34.155670 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5245c46-a520-45f9-b154-c0cb33d5eff7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:34.157090 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5245c46-a520-45f9-b154-c0cb33d5eff7 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:34.162469 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 396/777] 10.222.0.26 () {56 vars in 1112 bytes} [Mon May 9 09:34:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:34.167946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daae560-4cfc-4808-987e-112718383ba1 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:34.319431 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daae560-4cfc-4808-987e-112718383ba1 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:34.369458 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61602ba3-8053-4b2d-8475-f40f9fee5101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:34.370046 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61602ba3-8053-4b2d-8475-f40f9fee5101 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:34.370630 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61602ba3-8053-4b2d-8475-f40f9fee5101 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:34.370893 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61602ba3-8053-4b2d-8475-f40f9fee5101 None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:34.371599 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61602ba3-8053-4b2d-8475-f40f9fee5101 None admin] RBAC: Adding query filter params (type=network) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:34.371900 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61602ba3-8053-4b2d-8475-f40f9fee5101 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:34.373295 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61602ba3-8053-4b2d-8475-f40f9fee5101 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:34.378534 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 382/778] 10.222.0.26 () {56 vars in 1112 bytes} [Mon May 9 09:34:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:34.384515 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5245c46-a520-45f9-b154-c0cb33d5eff7 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:34.519750 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5245c46-a520-45f9-b154-c0cb33d5eff7 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:34.562716 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aed911-9033-42f3-84dd-becd0d670276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:34.563389 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aed911-9033-42f3-84dd-becd0d670276 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:34.563661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aed911-9033-42f3-84dd-becd0d670276 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:34.563925 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aed911-9033-42f3-84dd-becd0d670276 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:34.564735 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aed911-9033-42f3-84dd-becd0d670276 None admin] RBAC: Adding query filter params (interface=public, service_id=0d656827471945f0b6fd1848abf27d36, region_id=RegionOne) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:34.565040 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aed911-9033-42f3-84dd-becd0d670276 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:34.566400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aed911-9033-42f3-84dd-becd0d670276 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:34.579939 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 397/779] 10.222.0.26 () {56 vars in 1251 bytes} [Mon May 9 09:34:34 2022] GET /identity/v3/endpoints?service_id=0d656827471945f0b6fd1848abf27d36&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:34.585418 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61602ba3-8053-4b2d-8475-f40f9fee5101 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:34.712111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967522e-d910-46e3-a896-b41a7fe2eda7 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:34.712305 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61602ba3-8053-4b2d-8475-f40f9fee5101 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:34.712778 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967522e-d910-46e3-a896-b41a7fe2eda7 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:34.712778 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967522e-d910-46e3-a896-b41a7fe2eda7 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:34.715383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 398/780] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:34:34.727429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575b820-3a77-49ef-bb4a-112af5cfa4c1 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:34.727784 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575b820-3a77-49ef-bb4a-112af5cfa4c1 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:34.728041 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575b820-3a77-49ef-bb4a-112af5cfa4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:34.748158 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d58c05-843d-48bb-9021-897dfcc24719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:34.748862 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d58c05-843d-48bb-9021-897dfcc24719 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:34.749142 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d58c05-843d-48bb-9021-897dfcc24719 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:34.749417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d58c05-843d-48bb-9021-897dfcc24719 None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:34.750139 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d58c05-843d-48bb-9021-897dfcc24719 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:34.751578 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d58c05-843d-48bb-9021-897dfcc24719 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:34.757108 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 383/781] 10.222.0.26 () {56 vars in 1087 bytes} [Mon May 9 09:34:34 2022] GET /identity/v3/services => generated 772 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:34.758021 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-b575b820-3a77-49ef-bb4a-112af5cfa4c1 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:34.759288 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b575b820-3a77-49ef-bb4a-112af5cfa4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:34.883218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 399/782] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 09:34:34.919955 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d58c05-843d-48bb-9021-897dfcc24719 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:35.059285 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d58c05-843d-48bb-9021-897dfcc24719 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:35.109717 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d33c08-9924-45d0-9feb-7772f57987c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:35.110561 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d33c08-9924-45d0-9feb-7772f57987c0 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:35.110836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d33c08-9924-45d0-9feb-7772f57987c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:35.111100 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d33c08-9924-45d0-9feb-7772f57987c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:35.121780 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e1a03-0a19-4813-b20b-0ceb571809e6 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:35.122092 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e1a03-0a19-4813-b20b-0ceb571809e6 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:35.122331 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e1a03-0a19-4813-b20b-0ceb571809e6 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:35.123708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 400/783] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:34:35.129435 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e266e18-5938-44f8-b388-cf581f4d52f9 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:35.129720 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e266e18-5938-44f8-b388-cf581f4d52f9 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:35.129993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e266e18-5938-44f8-b388-cf581f4d52f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:35.134753 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d33c08-9924-45d0-9feb-7772f57987c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:35.142334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-92d33c08-9924-45d0-9feb-7772f57987c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:34:35.142334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:35.142334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:35.142334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:35.142334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:35.142334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:35.142334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:35.142334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:35.142334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:35.142334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:35.142334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:35.142334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:35.142334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:35.142334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:35.142334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:35.142334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:35.143400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:35.143400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:35.143400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:35.143400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:35.143400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:35.143400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:35.143400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:35.143400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:35.143400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:35.143400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:35.143400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:35.143400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:35.143400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:35.143400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:35.143400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:35.143400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:35.144351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:35.144351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:35.144351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:35.144351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:35.144351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:35.144351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:35.144351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:35.144351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:35.144351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:35.144351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:35.144351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:34:35.144351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:35.144351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 384/784] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:34:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:35.152195 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-5e266e18-5938-44f8-b388-cf581f4d52f9 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:35.153488 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-5e266e18-5938-44f8-b388-cf581f4d52f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:35.153683 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d33c08-9924-45d0-9feb-7772f57987c0 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:35.230555 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 401/785] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:34:35.257594 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e266e18-5938-44f8-b388-cf581f4d52f9 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:35.279974 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d33c08-9924-45d0-9feb-7772f57987c0 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:35.334484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baccc5f-8e68-4b89-9c4a-d0caa222c1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:35.335070 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baccc5f-8e68-4b89-9c4a-d0caa222c1b3 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:35.335344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baccc5f-8e68-4b89-9c4a-d0caa222c1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:35.335610 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baccc5f-8e68-4b89-9c4a-d0caa222c1b3 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:35.336357 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baccc5f-8e68-4b89-9c4a-d0caa222c1b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:35.336674 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baccc5f-8e68-4b89-9c4a-d0caa222c1b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:35.338097 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baccc5f-8e68-4b89-9c4a-d0caa222c1b3 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:35.355901 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 385/786] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:34:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:35.365703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baccc5f-8e68-4b89-9c4a-d0caa222c1b3 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:35.382278 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e266e18-5938-44f8-b388-cf581f4d52f9 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:35.416908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6e060-1bb0-4c9e-8803-954fcc0c824f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:35.417988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6e060-1bb0-4c9e-8803-954fcc0c824f None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:35.418417 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6e060-1bb0-4c9e-8803-954fcc0c824f None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:35.418846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6e060-1bb0-4c9e-8803-954fcc0c824f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:35.429135 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d6e060-1bb0-4c9e-8803-954fcc0c824f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:35.437017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-50d6e060-1bb0-4c9e-8803-954fcc0c824f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:34:35.437017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:35.437017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:35.437017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:35.437017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:35.437017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:35.437017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:35.437017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:35.437017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:35.437017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:35.437017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:35.437017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:35.437017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:35.437017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:35.437017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:35.437017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:35.438180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:35.438180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:35.438180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:35.438180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:35.438180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:35.438180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:35.438180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:35.438180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:35.438180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:35.438180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:35.438180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:35.438180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:35.438180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:35.438180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:35.438180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:35.438180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:35.439212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:35.439212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:35.439212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:35.439212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:35.439212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:35.439212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:35.439212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:35.439212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:35.439212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:35.439212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:35.439212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:34:35.439212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:35.441454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 402/787] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:34:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:35.449570 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6e060-1bb0-4c9e-8803-954fcc0c824f None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:35.518222 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baccc5f-8e68-4b89-9c4a-d0caa222c1b3 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:35.568691 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed62748-e16b-4781-ac3e-25cabd5c3c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:35.568691 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed62748-e16b-4781-ac3e-25cabd5c3c46 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:35.568950 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6e060-1bb0-4c9e-8803-954fcc0c824f None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:35.569043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed62748-e16b-4781-ac3e-25cabd5c3c46 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:35.569043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed62748-e16b-4781-ac3e-25cabd5c3c46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:35.575656 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed62748-e16b-4781-ac3e-25cabd5c3c46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:35.577163 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed62748-e16b-4781-ac3e-25cabd5c3c46 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:35.594251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-eed62748-e16b-4781-ac3e-25cabd5c3c46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:35.594251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:35.594251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:35.594251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:35.594251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:35.594251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:35.594251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:35.594251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:35.594251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:35.594251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:35.594251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:35.594251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:35.594251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:35.594251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:35.594251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:35.595185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:35.595185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:35.595185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:35.595185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:35.595185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:35.595185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:35.595185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:35.595185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:35.595185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:35.595185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:35.595185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:35.595185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:35.595185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:35.595185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:35.595185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:35.595185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:35.595185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:35.596098 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:35.596098 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:35.596098 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:35.596098 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:35.596098 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 386/788] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:35.600246 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202282e-0645-4630-a9d6-3e646987e17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:35.600246 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202282e-0645-4630-a9d6-3e646987e17d None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:35.600246 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202282e-0645-4630-a9d6-3e646987e17d None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:35.600246 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202282e-0645-4630-a9d6-3e646987e17d None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:35.600605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202282e-0645-4630-a9d6-3e646987e17d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:35.600605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202282e-0645-4630-a9d6-3e646987e17d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:35.601629 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed62748-e16b-4781-ac3e-25cabd5c3c46 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:35.609503 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202282e-0645-4630-a9d6-3e646987e17d None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:35.623516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 403/789] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:34:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:35.624836 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202282e-0645-4630-a9d6-3e646987e17d None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:35.729320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202282e-0645-4630-a9d6-3e646987e17d None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:35.764957 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb217f3f-f316-4c68-bcb6-13b3dd488b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:35.765751 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb217f3f-f316-4c68-bcb6-13b3dd488b75 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:35.766023 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb217f3f-f316-4c68-bcb6-13b3dd488b75 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:35.766291 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb217f3f-f316-4c68-bcb6-13b3dd488b75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:35.777894 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb217f3f-f316-4c68-bcb6-13b3dd488b75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:35.779610 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb217f3f-f316-4c68-bcb6-13b3dd488b75 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:35.798334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-fb217f3f-f316-4c68-bcb6-13b3dd488b75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:35.798334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:35.798334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:35.798334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:35.798334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:35.798334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:35.798334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:35.798334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:35.798334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:35.798334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:35.798334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:35.798334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:35.798334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:35.798334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:35.798334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:35.800539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:35.800539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:35.800539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:35.800539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:35.800539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:35.800539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:35.800539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:35.800539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:35.800539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:35.800539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:35.800539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:35.800539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:35.800539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:35.800539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:35.800539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:35.800539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:35.800539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:35.802843 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:35.802843 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:35.802843 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:35.802843 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:35.808703 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 404/790] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:35.808703 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb217f3f-f316-4c68-bcb6-13b3dd488b75 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:35.898056 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed62748-e16b-4781-ac3e-25cabd5c3c46 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:35.915991 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb217f3f-f316-4c68-bcb6-13b3dd488b75 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:35.958405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0f4c7-582b-41e6-9bdf-b44dbd8222bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:35.959085 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0f4c7-582b-41e6-9bdf-b44dbd8222bf None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:35.959368 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0f4c7-582b-41e6-9bdf-b44dbd8222bf None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:35.959645 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0f4c7-582b-41e6-9bdf-b44dbd8222bf None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:35.960391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0f4c7-582b-41e6-9bdf-b44dbd8222bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:35.960716 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0f4c7-582b-41e6-9bdf-b44dbd8222bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:35.962041 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b8baf-b0b3-4cd9-9363-0178a603f852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:35.962703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b8baf-b0b3-4cd9-9363-0178a603f852 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:35.962977 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b8baf-b0b3-4cd9-9363-0178a603f852 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:35.963246 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b8baf-b0b3-4cd9-9363-0178a603f852 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:35.963966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b8baf-b0b3-4cd9-9363-0178a603f852 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:35.964268 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b8baf-b0b3-4cd9-9363-0178a603f852 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:35.965675 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b8baf-b0b3-4cd9-9363-0178a603f852 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:35.968080 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0f4c7-582b-41e6-9bdf-b44dbd8222bf None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:35.987049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 405/791] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:35.989235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 387/792] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:35.992746 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0f4c7-582b-41e6-9bdf-b44dbd8222bf None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:35.994468 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b8baf-b0b3-4cd9-9363-0178a603f852 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:36.125548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0f4c7-582b-41e6-9bdf-b44dbd8222bf None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:36.135510 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b8baf-b0b3-4cd9-9363-0178a603f852 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:36.174693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e8b25-999f-4757-b57b-17af708e52ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:36.175934 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e8b25-999f-4757-b57b-17af708e52ef None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:36.176449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e8b25-999f-4757-b57b-17af708e52ef None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:36.176991 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e8b25-999f-4757-b57b-17af708e52ef None admin] PATH_INFO: `/v3/users/nova` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:36.177127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ad804-d4a9-47c8-92b7-f1a88cd13c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:36.178588 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ad804-d4a9-47c8-92b7-f1a88cd13c6f None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:36.179056 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ad804-d4a9-47c8-92b7-f1a88cd13c6f None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:36.179508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ad804-d4a9-47c8-92b7-f1a88cd13c6f None admin] PATH_INFO: `/v3/users/swift` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:36.190664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ad804-d4a9-47c8-92b7-f1a88cd13c6f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:36.191865 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ad804-d4a9-47c8-92b7-f1a88cd13c6f None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:36.192282 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e8b25-999f-4757-b57b-17af708e52ef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:36.194417 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e8b25-999f-4757-b57b-17af708e52ef None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:36.209679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-b08ad804-d4a9-47c8-92b7-f1a88cd13c6f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 09:34:36.209679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:36.209679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:36.209679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:36.209679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:36.209679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:36.209679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:36.209679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:36.209679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:36.209679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:36.209679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:36.209679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:36.209679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:36.209679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:36.209679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:36.209679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:36.210707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:36.210707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:36.210707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:36.210707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:36.210707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:36.210707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:36.210707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:36.210707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:36.210707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:36.210707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:36.210707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:36.210707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:36.210707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:36.210707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:36.210707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:36.210707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:36.210707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:36.210707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:36.211670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:36.211670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:36.211670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:36.211670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:36.211670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:36.211670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:36.211670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:36.211670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:36.211670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 09:34:36.211670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:36.211670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 388/793] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:34:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:36.214003 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc8190-506a-4751-84de-744ba689c8c0 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:36.214003 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc8190-506a-4751-84de-744ba689c8c0 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:36.214003 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc8190-506a-4751-84de-744ba689c8c0 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:36.218437 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 389/794] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:36.218437 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc8190-506a-4751-84de-744ba689c8c0 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:36.219459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-e20e8b25-999f-4757-b57b-17af708e52ef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 09:34:36.219459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:36.219459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:36.219459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:36.219459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:36.219459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:36.219459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:36.219459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:36.219459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:36.219459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:36.219459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:36.219459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:36.219459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:36.219459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:36.219459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:36.219459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:36.220398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:36.220398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:36.220398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:36.220398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:36.220398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:36.220398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:36.220398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:36.220398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:36.220398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:36.220398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:36.220398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:36.220398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:36.220398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:36.220398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:36.220398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:36.220398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:36.220398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:36.220398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:36.221382 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:36.221382 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:36.221382 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:36.221382 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:36.221382 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:36.221382 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:36.221382 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:36.221382 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:36.221382 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 09:34:36.221382 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:36.225112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 406/795] 10.222.0.26 () {56 vars in 1093 bytes} [Mon May 9 09:34:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:36.226586 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc802d-f343-424a-a05e-77bc9c78ed4e None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:36.227089 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc802d-f343-424a-a05e-77bc9c78ed4e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:36.227633 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc802d-f343-424a-a05e-77bc9c78ed4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:36.257037 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-46fc802d-f343-424a-a05e-77bc9c78ed4e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:36.258219 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-46fc802d-f343-424a-a05e-77bc9c78ed4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:36.335413 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc8190-506a-4751-84de-744ba689c8c0 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:36.336290 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 407/796] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:34:36.337613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc802d-f343-424a-a05e-77bc9c78ed4e None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:36.376142 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145dcfa-ce15-4724-a6f0-0c5ca8029079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:36.376846 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145dcfa-ce15-4724-a6f0-0c5ca8029079 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:36.377148 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145dcfa-ce15-4724-a6f0-0c5ca8029079 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:36.377404 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145dcfa-ce15-4724-a6f0-0c5ca8029079 None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:36.378255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145dcfa-ce15-4724-a6f0-0c5ca8029079 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:36.378562 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145dcfa-ce15-4724-a6f0-0c5ca8029079 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:36.379990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145dcfa-ce15-4724-a6f0-0c5ca8029079 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:36.406716 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 390/797] 10.222.0.26 () {56 vars in 1135 bytes} [Mon May 9 09:34:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:36.406716 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1de7f-e69f-4ffa-8f5f-eb1fadda97c4 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:36.406716 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1de7f-e69f-4ffa-8f5f-eb1fadda97c4 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:36.406716 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1de7f-e69f-4ffa-8f5f-eb1fadda97c4 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:36.407268 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 391/798] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:36.407826 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1de7f-e69f-4ffa-8f5f-eb1fadda97c4 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:36.444140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc802d-f343-424a-a05e-77bc9c78ed4e None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:36.481114 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c4f23-947f-4843-8cc1-788eec25db7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:36.481795 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c4f23-947f-4843-8cc1-788eec25db7d None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:36.482072 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c4f23-947f-4843-8cc1-788eec25db7d None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:36.482343 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c4f23-947f-4843-8cc1-788eec25db7d None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:36.483183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c4f23-947f-4843-8cc1-788eec25db7d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:36.483486 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c4f23-947f-4843-8cc1-788eec25db7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:36.484950 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c4f23-947f-4843-8cc1-788eec25db7d None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:36.502335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 408/799] 10.222.0.26 () {56 vars in 1133 bytes} [Mon May 9 09:34:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:36.503135 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c4f23-947f-4843-8cc1-788eec25db7d None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:36.511679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1de7f-e69f-4ffa-8f5f-eb1fadda97c4 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:36.550182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03d769-3601-441c-aa8e-b018ef0da293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:36.550973 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03d769-3601-441c-aa8e-b018ef0da293 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:36.551242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03d769-3601-441c-aa8e-b018ef0da293 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:36.551501 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03d769-3601-441c-aa8e-b018ef0da293 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:36.566762 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03d769-3601-441c-aa8e-b018ef0da293 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:36.574416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-2b03d769-3601-441c-aa8e-b018ef0da293 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:36.574416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:36.574416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:36.574416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:36.574416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:36.574416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:36.574416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:36.574416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:36.574416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:36.574416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:36.574416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:36.574416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:36.574416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:36.574416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:36.574416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:36.574416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:36.575373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:36.575373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:36.575373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:36.575373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:36.575373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:36.575373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:36.575373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:36.575373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:36.575373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:36.575373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:36.575373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:36.575373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:36.575373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:36.575373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:36.575373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:36.575373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:36.576235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:36.576235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:36.576235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:36.576235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:36.576235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:36.576235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:36.576235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:36.576235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:36.576235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:36.576235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:36.576235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:36.576235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:36.576235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 392/800] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:34:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:36.577458 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b4bf4-075d-4938-a2db-d0992fa2ecbb None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:36.577570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b4bf4-075d-4938-a2db-d0992fa2ecbb None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:36.577809 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b4bf4-075d-4938-a2db-d0992fa2ecbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:36.594970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-677b4bf4-075d-4938-a2db-d0992fa2ecbb None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:36.596341 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-677b4bf4-075d-4938-a2db-d0992fa2ecbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:36.603648 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c4f23-947f-4843-8cc1-788eec25db7d None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:36.636589 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd26d1-e001-442b-bba8-b9e33d6038c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:36.637426 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd26d1-e001-442b-bba8-b9e33d6038c0 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:36.637710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd26d1-e001-442b-bba8-b9e33d6038c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:36.637984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd26d1-e001-442b-bba8-b9e33d6038c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:36.648001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd26d1-e001-442b-bba8-b9e33d6038c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:36.649568 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd26d1-e001-442b-bba8-b9e33d6038c0 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:36.668686 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 393/801] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:34:36.668686 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b4bf4-075d-4938-a2db-d0992fa2ecbb None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:36.680234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-eacd26d1-e001-442b-bba8-b9e33d6038c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:36.680234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:36.680234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:36.680234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:36.680234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:36.680234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:36.680234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:36.680234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:36.680234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:36.680234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:36.680234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:36.680234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:36.680234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:36.680234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:36.680234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:36.681530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:36.681530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:36.681530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:36.681530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:36.681530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:36.681530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:36.681530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:36.681530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:36.681530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:36.681530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:36.681530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:36.681530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:36.681530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:36.681530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:36.681530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:36.681530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:36.681530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:36.682746 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:36.682746 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:36.682746 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:36.682746 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:36.682746 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 409/802] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:36.684054 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd26d1-e001-442b-bba8-b9e33d6038c0 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:36.752001 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b4bf4-075d-4938-a2db-d0992fa2ecbb None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:36.785961 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61784c-bd66-4779-a23c-0cce643cf732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:36.786612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61784c-bd66-4779-a23c-0cce643cf732 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:36.786888 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61784c-bd66-4779-a23c-0cce643cf732 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:36.787157 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61784c-bd66-4779-a23c-0cce643cf732 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:36.787257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd26d1-e001-442b-bba8-b9e33d6038c0 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:36.818407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470df735-3d1a-407a-85a9-a4b435b5d6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:36.819067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470df735-3d1a-407a-85a9-a4b435b5d6b7 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:36.819340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470df735-3d1a-407a-85a9-a4b435b5d6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:36.819608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470df735-3d1a-407a-85a9-a4b435b5d6b7 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:36.820345 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470df735-3d1a-407a-85a9-a4b435b5d6b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:36.820800 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470df735-3d1a-407a-85a9-a4b435b5d6b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:36.822224 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470df735-3d1a-407a-85a9-a4b435b5d6b7 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:36.834335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 410/803] 10.222.0.26 () {56 vars in 1103 bytes} [Mon May 9 09:34:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:36.834335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470df735-3d1a-407a-85a9-a4b435b5d6b7 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:36.856516 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61784c-bd66-4779-a23c-0cce643cf732 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:36.858008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61784c-bd66-4779-a23c-0cce643cf732 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:36.931119 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470df735-3d1a-407a-85a9-a4b435b5d6b7 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:36.952081 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 394/804] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:34:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 09:34:36.952797 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61784c-bd66-4779-a23c-0cce643cf732 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:36.971777 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497cebdb-2a6a-4422-bc8e-6f79cf000232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:36.972611 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497cebdb-2a6a-4422-bc8e-6f79cf000232 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:36.972930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497cebdb-2a6a-4422-bc8e-6f79cf000232 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:36.973490 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497cebdb-2a6a-4422-bc8e-6f79cf000232 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:36.973852 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497cebdb-2a6a-4422-bc8e-6f79cf000232 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:36.975230 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497cebdb-2a6a-4422-bc8e-6f79cf000232 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:36.987274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-497cebdb-2a6a-4422-bc8e-6f79cf000232 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 09:34:36.987274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:36.987274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:34:36.987274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:34:36.987274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:34:36.987274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:34:36.987274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:34:36.987274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:36.987274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 09:34:36.987274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:36.987274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:34:36.987274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:36.987274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:36.987274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:36.987274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:36.987274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:36.988225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:36.988225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:36.988225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:36.988225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:36.988225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:36.988225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:36.988225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:36.988225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:34:36.988225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:34:36.988225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:34:36.988225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:34:36.988225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:36.988225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:36.988225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:36.988225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:36.988225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:36.989147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:36.989147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:36.989147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:36.989147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:36.989147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:36.989147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:36.989147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:36.989147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:36.989147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:36.989147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:36.989147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:36.989147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:34:36.989147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:36.989147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 09:34:36.989147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:36.993044 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 411/805] 10.222.0.26 () {56 vars in 1129 bytes} [Mon May 9 09:34:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:36.994118 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497cebdb-2a6a-4422-bc8e-6f79cf000232 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:37.054240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61784c-bd66-4779-a23c-0cce643cf732 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:37.086996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12575a24-1418-4737-835e-47650663a6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:37.087656 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12575a24-1418-4737-835e-47650663a6be None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:37.087939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12575a24-1418-4737-835e-47650663a6be None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:37.088199 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12575a24-1418-4737-835e-47650663a6be None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:37.088946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12575a24-1418-4737-835e-47650663a6be None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:37.089247 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12575a24-1418-4737-835e-47650663a6be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:37.090525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497cebdb-2a6a-4422-bc8e-6f79cf000232 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:37.090632 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12575a24-1418-4737-835e-47650663a6be None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:37.106303 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 395/806] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:37.107172 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12575a24-1418-4737-835e-47650663a6be None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:37.122830 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2dff2-99fa-4411-8259-aad4e21efa8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:37.123627 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2dff2-99fa-4411-8259-aad4e21efa8c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:37.123906 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2dff2-99fa-4411-8259-aad4e21efa8c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:37.124199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2dff2-99fa-4411-8259-aad4e21efa8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:37.134914 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef2dff2-99fa-4411-8259-aad4e21efa8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:37.136360 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef2dff2-99fa-4411-8259-aad4e21efa8c None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:37.153209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-2ef2dff2-99fa-4411-8259-aad4e21efa8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:37.153209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:37.153209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:37.153209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:37.153209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:37.153209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:37.153209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:37.153209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:37.153209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:37.153209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:37.153209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:37.153209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:37.153209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:37.153209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:37.153209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:37.154167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:37.154167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:37.154167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:37.154167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:37.154167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:37.154167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:37.154167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:37.154167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:37.154167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:37.154167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:37.154167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:37.154167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:37.154167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:37.154167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:37.154167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:37.154167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:37.154167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:37.155054 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:37.155054 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:37.155054 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:37.155054 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:37.159066 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 412/807] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:37.160207 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2dff2-99fa-4411-8259-aad4e21efa8c None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:37.203822 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12575a24-1418-4737-835e-47650663a6be None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:37.239769 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a5f3a-9007-4d6f-893c-f7cc83c21f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:37.240582 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a5f3a-9007-4d6f-893c-f7cc83c21f3b None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:37.240888 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a5f3a-9007-4d6f-893c-f7cc83c21f3b None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:37.241161 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a5f3a-9007-4d6f-893c-f7cc83c21f3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:37.252748 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a5f3a-9007-4d6f-893c-f7cc83c21f3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:37.254232 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a5f3a-9007-4d6f-893c-f7cc83c21f3b None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:37.271207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-607a5f3a-9007-4d6f-893c-f7cc83c21f3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:37.271207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:37.271207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:37.271207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:37.271207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:37.271207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:37.271207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:37.271207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:37.271207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:37.271207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:37.271207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:37.271207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:37.271207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:37.271207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:37.271207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:37.272250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:37.272250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:37.272250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:37.272250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:37.272250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:37.272250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:37.272250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:37.272250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:37.272250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:37.272250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:37.272250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:37.272250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:37.272250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:37.272250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:37.272250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:37.272250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:37.272250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:37.273216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:37.273216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:37.273216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:37.273216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:37.273454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2dff2-99fa-4411-8259-aad4e21efa8c None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:37.273530 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 396/808] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:37.273530 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a5f3a-9007-4d6f-893c-f7cc83c21f3b None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:37.308865 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d953e41-ac55-404a-8fa0-833f3595f7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:37.309455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d953e41-ac55-404a-8fa0-833f3595f7fd None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:37.309730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d953e41-ac55-404a-8fa0-833f3595f7fd None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:37.310004 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d953e41-ac55-404a-8fa0-833f3595f7fd None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:37.310700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d953e41-ac55-404a-8fa0-833f3595f7fd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:37.310999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d953e41-ac55-404a-8fa0-833f3595f7fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:37.312364 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d953e41-ac55-404a-8fa0-833f3595f7fd None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:37.321582 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 413/809] 10.222.0.26 () {56 vars in 1124 bytes} [Mon May 9 09:34:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:37.321582 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d953e41-ac55-404a-8fa0-833f3595f7fd None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:37.384090 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a5f3a-9007-4d6f-893c-f7cc83c21f3b None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:37.416771 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c452b-80d3-4b9e-9a42-ce8605036d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:37.417562 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c452b-80d3-4b9e-9a42-ce8605036d2b None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:37.417722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d953e41-ac55-404a-8fa0-833f3595f7fd None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:37.417822 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c452b-80d3-4b9e-9a42-ce8605036d2b None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:37.418153 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c452b-80d3-4b9e-9a42-ce8605036d2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:37.424615 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c452b-80d3-4b9e-9a42-ce8605036d2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:37.426069 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c452b-80d3-4b9e-9a42-ce8605036d2b None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:37.439607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-a64c452b-80d3-4b9e-9a42-ce8605036d2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:37.439607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:37.439607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:37.439607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:37.439607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:37.439607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:37.439607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:37.439607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:37.439607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:37.439607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:37.439607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:37.439607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:37.439607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:37.439607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:37.439607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:37.440505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:37.440505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:37.440505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:37.440505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:37.440505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:37.440505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:37.440505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:37.440505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:37.440505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:37.440505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:37.440505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:37.440505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:37.440505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:37.440505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:37.440505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:37.440505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:37.440505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:37.442043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:37.442043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:37.442043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:37.442043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:37.442043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:37.442043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:37.442043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:37.442043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:37.442043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 397/810] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:37.442043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c452b-80d3-4b9e-9a42-ce8605036d2b None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:37.450941 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adaeb3c-767c-4dc3-99eb-5cc54db6ea96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:37.451606 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adaeb3c-767c-4dc3-99eb-5cc54db6ea96 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:37.451884 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adaeb3c-767c-4dc3-99eb-5cc54db6ea96 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:37.452155 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adaeb3c-767c-4dc3-99eb-5cc54db6ea96 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:37.452923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adaeb3c-767c-4dc3-99eb-5cc54db6ea96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:37.453228 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adaeb3c-767c-4dc3-99eb-5cc54db6ea96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:37.454595 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adaeb3c-767c-4dc3-99eb-5cc54db6ea96 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:37.466321 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 414/811] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:37.467122 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adaeb3c-767c-4dc3-99eb-5cc54db6ea96 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:37.535894 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c452b-80d3-4b9e-9a42-ce8605036d2b None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:37.560757 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adaeb3c-767c-4dc3-99eb-5cc54db6ea96 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:37.572542 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d77d2e-0615-4532-9955-d9dcdd56729e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:37.573268 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d77d2e-0615-4532-9955-d9dcdd56729e None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:37.573544 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d77d2e-0615-4532-9955-d9dcdd56729e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:37.573815 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d77d2e-0615-4532-9955-d9dcdd56729e None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:37.574536 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d77d2e-0615-4532-9955-d9dcdd56729e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:37.574839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d77d2e-0615-4532-9955-d9dcdd56729e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:37.576206 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d77d2e-0615-4532-9955-d9dcdd56729e None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:37.587218 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 398/812] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:37.588410 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6560b-08fe-4ef8-870b-c1a2015c9bd6 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:37.588699 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6560b-08fe-4ef8-870b-c1a2015c9bd6 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:37.588971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6560b-08fe-4ef8-870b-c1a2015c9bd6 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:37.590467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 399/813] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:37.591166 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6560b-08fe-4ef8-870b-c1a2015c9bd6 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:37.594931 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f4333-94cb-45ac-8295-7d5f5f31cbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:37.595587 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f4333-94cb-45ac-8295-7d5f5f31cbaa None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:37.595902 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f4333-94cb-45ac-8295-7d5f5f31cbaa None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:37.596171 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f4333-94cb-45ac-8295-7d5f5f31cbaa None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:37.596904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7f4333-94cb-45ac-8295-7d5f5f31cbaa None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:37.597203 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7f4333-94cb-45ac-8295-7d5f5f31cbaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:37.598553 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7f4333-94cb-45ac-8295-7d5f5f31cbaa None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:37.603364 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 415/814] 10.222.0.26 () {56 vars in 1124 bytes} [Mon May 9 09:34:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:37.604112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f4333-94cb-45ac-8295-7d5f5f31cbaa None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:37.689000 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6560b-08fe-4ef8-870b-c1a2015c9bd6 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:37.694401 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f4333-94cb-45ac-8295-7d5f5f31cbaa None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:37.722397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc07ec3-9eee-4b43-a702-a7c863661c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:37.723124 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc07ec3-9eee-4b43-a702-a7c863661c09 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:37.723403 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc07ec3-9eee-4b43-a702-a7c863661c09 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:37.723665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc07ec3-9eee-4b43-a702-a7c863661c09 None admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:37.724449 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc07ec3-9eee-4b43-a702-a7c863661c09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:37.724782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc07ec3-9eee-4b43-a702-a7c863661c09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:37.726205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc07ec3-9eee-4b43-a702-a7c863661c09 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:37.726611 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c715b-0194-418f-851a-f58f4a1ea8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:37.727472 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c715b-0194-418f-851a-f58f4a1ea8bf None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:37.727759 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c715b-0194-418f-851a-f58f4a1ea8bf None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:37.728021 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c715b-0194-418f-851a-f58f4a1ea8bf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:37.734788 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c715b-0194-418f-851a-f58f4a1ea8bf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:37.736203 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c715b-0194-418f-851a-f58f4a1ea8bf None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:37.737899 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 400/815] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:37.738658 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc07ec3-9eee-4b43-a702-a7c863661c09 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:37.752269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-9c7c715b-0194-418f-851a-f58f4a1ea8bf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 09:34:37.752269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:37.752269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:37.752269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:37.752269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:37.752269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:37.752269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:37.752269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:37.752269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:37.752269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:37.752269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:37.752269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:37.752269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:37.752269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:37.752269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:37.752269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:37.753357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:37.753357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:37.753357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:37.753357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:37.753357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:37.753357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:37.753357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:37.753357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:37.753357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:37.753357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:37.753357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:37.753357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:37.753357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:37.753357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:37.753357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:37.753357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:37.753357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:37.753357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:37.754327 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:37.754327 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:37.754327 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:37.754327 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:37.754327 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:37.754327 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:37.754327 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:37.754327 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:37.754327 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 09:34:37.754327 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:37.754327 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 416/816] 10.222.0.26 () {56 vars in 1117 bytes} [Mon May 9 09:34:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:37.754959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88202376-1121-43cf-b449-f7b4e327d33d None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:37.754959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88202376-1121-43cf-b449-f7b4e327d33d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:37.755378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88202376-1121-43cf-b449-f7b4e327d33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:37.770799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-88202376-1121-43cf-b449-f7b4e327d33d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:37.771972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-88202376-1121-43cf-b449-f7b4e327d33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:37.833016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc07ec3-9eee-4b43-a702-a7c863661c09 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:37.833567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 417/817] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:34:37.834341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88202376-1121-43cf-b449-f7b4e327d33d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:37.866907 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94447fdf-2c26-4a42-9dae-e7843557a6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:37.867709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94447fdf-2c26-4a42-9dae-e7843557a6b3 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:37.867985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94447fdf-2c26-4a42-9dae-e7843557a6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:37.868259 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94447fdf-2c26-4a42-9dae-e7843557a6b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:37.875160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94447fdf-2c26-4a42-9dae-e7843557a6b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:37.876617 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94447fdf-2c26-4a42-9dae-e7843557a6b3 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:37.890094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-94447fdf-2c26-4a42-9dae-e7843557a6b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:37.890094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:37.890094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:37.890094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:37.890094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:37.890094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:37.890094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:37.890094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:37.890094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:37.890094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:37.890094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:37.890094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:37.890094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:37.890094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:37.890094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:37.891167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:37.891167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:37.891167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:37.891167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:37.891167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:37.891167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:37.891167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:37.891167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:37.891167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:37.891167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:37.891167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:37.891167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:37.891167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:37.891167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:37.891167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:37.891167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:37.891167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:37.892075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:37.892075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:37.892075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:37.892075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:37.892075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:37.892075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:37.892075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:37.892075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:37.892075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 401/818] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:37.892635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94447fdf-2c26-4a42-9dae-e7843557a6b3 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:37.931670 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88202376-1121-43cf-b449-f7b4e327d33d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:37.964917 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec3f02-4996-4420-a8c9-8b0076bd5262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:37.965558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec3f02-4996-4420-a8c9-8b0076bd5262 None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:37.965825 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec3f02-4996-4420-a8c9-8b0076bd5262 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:37.966089 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec3f02-4996-4420-a8c9-8b0076bd5262 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:37.966823 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec3f02-4996-4420-a8c9-8b0076bd5262 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:37.968161 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec3f02-4996-4420-a8c9-8b0076bd5262 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:37.985305 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94447fdf-2c26-4a42-9dae-e7843557a6b3 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:37.990463 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 418/819] 10.222.0.26 () {60 vars in 1144 bytes} [Mon May 9 09:34:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:34:37.991310 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec3f02-4996-4420-a8c9-8b0076bd5262 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:38.033903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669833af-8158-428e-9429-feb1856dd983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:38.034802 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669833af-8158-428e-9429-feb1856dd983 None admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:38.035079 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669833af-8158-428e-9429-feb1856dd983 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:38.035347 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669833af-8158-428e-9429-feb1856dd983 None admin] PATH_INFO: `/v3/projects/9cd39464055141b8b43fde20d0885c99/users/380e04358f2349aa912d6a97518877cd/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:38.037384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669833af-8158-428e-9429-feb1856dd983 None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd39464055141b8b43fde20d0885c99, user_id=380e04358f2349aa912d6a97518877cd, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:38.038834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669833af-8158-428e-9429-feb1856dd983 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:38.054271 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 402/820] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:34:37 2022] PUT /identity/v3/projects/9cd39464055141b8b43fde20d0885c99/users/380e04358f2349aa912d6a97518877cd/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:34:38.055330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669833af-8158-428e-9429-feb1856dd983 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:38.100991 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec3f02-4996-4420-a8c9-8b0076bd5262 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:38.138386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562c61d-564c-48a4-8b85-440b47edbc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:38.139067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562c61d-564c-48a4-8b85-440b47edbc46 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:38.139344 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562c61d-564c-48a4-8b85-440b47edbc46 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:38.139616 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562c61d-564c-48a4-8b85-440b47edbc46 None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:38.140483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562c61d-564c-48a4-8b85-440b47edbc46 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:38.140827 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562c61d-564c-48a4-8b85-440b47edbc46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:38.142353 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562c61d-564c-48a4-8b85-440b47edbc46 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:38.161932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 419/821] 10.222.0.26 () {56 vars in 1149 bytes} [Mon May 9 09:34:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:38.162824 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562c61d-564c-48a4-8b85-440b47edbc46 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:38.186968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669833af-8158-428e-9429-feb1856dd983 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:38.228724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04549e2-bbdd-4258-91c9-6159c7cea897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:38.229548 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04549e2-bbdd-4258-91c9-6159c7cea897 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:38.229825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04549e2-bbdd-4258-91c9-6159c7cea897 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:38.230258 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04549e2-bbdd-4258-91c9-6159c7cea897 None admin] PATH_INFO: `/v3/services/network` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:38.231315 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04549e2-bbdd-4258-91c9-6159c7cea897 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:38.232762 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04549e2-bbdd-4258-91c9-6159c7cea897 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:38.247146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-b04549e2-bbdd-4258-91c9-6159c7cea897 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 09:34:38.247146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:38.247146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:34:38.247146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:34:38.247146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:34:38.247146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:34:38.247146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:34:38.247146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:38.247146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 09:34:38.247146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:38.247146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:34:38.247146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:38.247146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:38.247146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:38.247146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:38.247146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:38.247146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:38.248205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:38.248205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:38.248205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:38.248205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:38.248205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:38.248205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:38.248205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:34:38.248205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:34:38.248205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:34:38.248205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:34:38.248205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:38.248205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:38.248205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:38.248205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:38.248205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:38.248205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:38.249482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:38.249482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:38.249482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:38.249482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:38.249482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:38.249482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:38.249482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:38.249482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:38.249482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:38.249482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:38.249482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:34:38.249482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:38.249482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 09:34:38.249482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:38.249482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 403/822] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:38 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:38.250512 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04549e2-bbdd-4258-91c9-6159c7cea897 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:38.285874 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562c61d-564c-48a4-8b85-440b47edbc46 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:38.328953 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663516b-27e4-486d-9488-f1a8cbd36481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:38.329605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663516b-27e4-486d-9488-f1a8cbd36481 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:38.329885 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663516b-27e4-486d-9488-f1a8cbd36481 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:38.330158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663516b-27e4-486d-9488-f1a8cbd36481 None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:38.330958 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4663516b-27e4-486d-9488-f1a8cbd36481 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:38.331264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4663516b-27e4-486d-9488-f1a8cbd36481 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:38.332735 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4663516b-27e4-486d-9488-f1a8cbd36481 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:38.345741 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 420/823] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:38.346588 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663516b-27e4-486d-9488-f1a8cbd36481 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:38.375442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04549e2-bbdd-4258-91c9-6159c7cea897 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:38.410916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49e77c-659e-48cc-8884-c152cfc5abe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:38.411804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49e77c-659e-48cc-8884-c152cfc5abe0 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:38.412151 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49e77c-659e-48cc-8884-c152cfc5abe0 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:38.412491 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49e77c-659e-48cc-8884-c152cfc5abe0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:38.423598 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49e77c-659e-48cc-8884-c152cfc5abe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:38.425241 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49e77c-659e-48cc-8884-c152cfc5abe0 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:38.442880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-ea49e77c-659e-48cc-8884-c152cfc5abe0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:38.442880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:38.442880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:38.442880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:38.442880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:38.442880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:38.442880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:38.442880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:38.442880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:38.442880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:38.442880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:38.442880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:38.442880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:38.442880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:38.442880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:38.443810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:38.443810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:38.443810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:38.443810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:38.443810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:38.443810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:38.443810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:38.443810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:38.443810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:38.443810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:38.443810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:38.443810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:38.443810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:38.443810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:38.443810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:38.443810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:38.443810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:38.445440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:38.445440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:38.445440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:38.445440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:38.445440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 404/824] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:38.446134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49e77c-659e-48cc-8884-c152cfc5abe0 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:38.456995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663516b-27e4-486d-9488-f1a8cbd36481 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:38.499421 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d05ec-3da9-4a41-abe8-19d8efae42e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:38.500091 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d05ec-3da9-4a41-abe8-19d8efae42e9 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:38.500523 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d05ec-3da9-4a41-abe8-19d8efae42e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:38.500824 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d05ec-3da9-4a41-abe8-19d8efae42e9 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:38.502108 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d05ec-3da9-4a41-abe8-19d8efae42e9 None admin] RBAC: Adding query filter params (name=network) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:38.502417 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d05ec-3da9-4a41-abe8-19d8efae42e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:38.504280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d05ec-3da9-4a41-abe8-19d8efae42e9 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:38.510260 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 421/825] 10.222.0.26 () {56 vars in 1112 bytes} [Mon May 9 09:34:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:38.511539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d05ec-3da9-4a41-abe8-19d8efae42e9 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:38.551367 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49e77c-659e-48cc-8884-c152cfc5abe0 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:38.591497 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79138786-b412-434d-be13-efcf1d7d9484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:38.592185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79138786-b412-434d-be13-efcf1d7d9484 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:38.592622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79138786-b412-434d-be13-efcf1d7d9484 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:38.592944 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79138786-b412-434d-be13-efcf1d7d9484 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:38.594416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79138786-b412-434d-be13-efcf1d7d9484 None admin] RBAC: Adding query filter params (role.id=70ac549382b14b2cbb8b35e5dfc648d4, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=6362303cb918486c8525225914332ad5) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:38.594727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79138786-b412-434d-be13-efcf1d7d9484 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:38.596643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79138786-b412-434d-be13-efcf1d7d9484 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:38.609118 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 405/826] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:34:38 2022] GET /identity/v3/role_assignments?user.id=6362303cb918486c8525225914332ad5&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=70ac549382b14b2cbb8b35e5dfc648d4 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:38.610125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79138786-b412-434d-be13-efcf1d7d9484 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:38.666245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d05ec-3da9-4a41-abe8-19d8efae42e9 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:38.756233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79138786-b412-434d-be13-efcf1d7d9484 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:38.762454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bdeb0-0f21-4ab0-b474-d14b46d865be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:38.763143 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bdeb0-0f21-4ab0-b474-d14b46d865be None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:38.763420 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bdeb0-0f21-4ab0-b474-d14b46d865be None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:38.763690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bdeb0-0f21-4ab0-b474-d14b46d865be None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:38.764422 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2bdeb0-0f21-4ab0-b474-d14b46d865be None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:38.764751 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2bdeb0-0f21-4ab0-b474-d14b46d865be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:38.766165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2bdeb0-0f21-4ab0-b474-d14b46d865be None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:38.792854 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f27789-82af-4def-8449-66bea9ceb72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:38.793534 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f27789-82af-4def-8449-66bea9ceb72a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:38.793813 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f27789-82af-4def-8449-66bea9ceb72a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:38.794079 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f27789-82af-4def-8449-66bea9ceb72a None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:38.794796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f27789-82af-4def-8449-66bea9ceb72a None admin] RBAC: Adding query filter params (type=network) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:38.795100 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f27789-82af-4def-8449-66bea9ceb72a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:38.796699 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f27789-82af-4def-8449-66bea9ceb72a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:38.799359 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 422/827] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:38.802655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 406/828] 10.222.0.26 () {56 vars in 1112 bytes} [Mon May 9 09:34:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:38.804724 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bdeb0-0f21-4ab0-b474-d14b46d865be None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:38.809723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f27789-82af-4def-8449-66bea9ceb72a None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:38.915545 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f27789-82af-4def-8449-66bea9ceb72a None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:38.957892 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bdeb0-0f21-4ab0-b474-d14b46d865be None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:38.960355 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66910a-9fef-49c3-a39d-5f26c28752bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:38.961047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66910a-9fef-49c3-a39d-5f26c28752bb None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:38.961320 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66910a-9fef-49c3-a39d-5f26c28752bb None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:38.961586 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66910a-9fef-49c3-a39d-5f26c28752bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:38.962409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66910a-9fef-49c3-a39d-5f26c28752bb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:38.963782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66910a-9fef-49c3-a39d-5f26c28752bb None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:38.999550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 407/829] 10.222.0.26 () {60 vars in 1144 bytes} [Mon May 9 09:34:38 2022] POST /identity/v3/endpoints => generated 327 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:34:39.031275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa367e-3de3-4295-a9f2-3552343d6173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:39.032084 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa367e-3de3-4295-a9f2-3552343d6173 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:39.032454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa367e-3de3-4295-a9f2-3552343d6173 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:39.032762 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa367e-3de3-4295-a9f2-3552343d6173 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:39.043560 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa367e-3de3-4295-a9f2-3552343d6173 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:39.045071 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa367e-3de3-4295-a9f2-3552343d6173 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:39.065024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-3faa367e-3de3-4295-a9f2-3552343d6173 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:39.065024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:39.065024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:39.065024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:39.065024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:39.065024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:39.065024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:39.065024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:39.065024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:39.065024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:39.065024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:39.065024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:39.065024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:39.065024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:39.065024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:39.065971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:39.065971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:39.065971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:39.065971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:39.065971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:39.065971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:39.065971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:39.065971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:39.065971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:39.065971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:39.065971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:39.065971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:39.065971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:39.065971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:39.065971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:39.065971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:39.065971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:39.066863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:39.066863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:39.066863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:39.066863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:39.066863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:39.066863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:39.066863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:39.066863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:39.072828 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 423/830] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:39.074528 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66910a-9fef-49c3-a39d-5f26c28752bb None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:39.210234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66910a-9fef-49c3-a39d-5f26c28752bb None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:39.255028 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f88e0-d7a5-4122-8334-f70a8fb5ef2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:39.255693 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f88e0-d7a5-4122-8334-f70a8fb5ef2d None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:39.255969 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f88e0-d7a5-4122-8334-f70a8fb5ef2d None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:39.256240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f88e0-d7a5-4122-8334-f70a8fb5ef2d None admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:39.257121 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f88e0-d7a5-4122-8334-f70a8fb5ef2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:39.257428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f88e0-d7a5-4122-8334-f70a8fb5ef2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:39.259054 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f88e0-d7a5-4122-8334-f70a8fb5ef2d None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:39.275704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 408/831] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:39.281192 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa367e-3de3-4295-a9f2-3552343d6173 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:39.421151 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa367e-3de3-4295-a9f2-3552343d6173 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:39.454868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdab5a5-9081-430a-9cb1-b22cfc224f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:39.455769 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdab5a5-9081-430a-9cb1-b22cfc224f73 None admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:39.456047 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdab5a5-9081-430a-9cb1-b22cfc224f73 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:39.456317 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdab5a5-9081-430a-9cb1-b22cfc224f73 None admin] PATH_INFO: `/v3/projects/9cd39464055141b8b43fde20d0885c99/users/af62b2d3f73f482b8db4173af33752c6/roles/f55c3728df78466989d0cd1b13c5ea28` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:39.472505 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdab5a5-9081-430a-9cb1-b22cfc224f73 None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd39464055141b8b43fde20d0885c99, user_id=af62b2d3f73f482b8db4173af33752c6, role_id=f55c3728df78466989d0cd1b13c5ea28)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:39.473923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdab5a5-9081-430a-9cb1-b22cfc224f73 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:39.491414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 424/832] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:34:39 2022] PUT /identity/v3/projects/9cd39464055141b8b43fde20d0885c99/users/af62b2d3f73f482b8db4173af33752c6/roles/f55c3728df78466989d0cd1b13c5ea28 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:34:40.485128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30577d3c-f855-414a-9356-597e6191ab05 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:40.486213 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30577d3c-f855-414a-9356-597e6191ab05 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:40.486697 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30577d3c-f855-414a-9356-597e6191ab05 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:40.488567 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 409/833] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:40.494209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5f48d-1c9a-4969-8134-fe074f18a36a None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:40.495025 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5f48d-1c9a-4969-8134-fe074f18a36a None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:40.495816 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5f48d-1c9a-4969-8134-fe074f18a36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:40.504155 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4eba0-a329-4380-a3ed-e58b95f91b2b None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:40.504797 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4eba0-a329-4380-a3ed-e58b95f91b2b None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:40.505271 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4eba0-a329-4380-a3ed-e58b95f91b2b None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:40.507077 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 410/834] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:40.514075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ec73f-ae53-43c2-801b-258d618fa465 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:40.514075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ec73f-ae53-43c2-801b-258d618fa465 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:40.514075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ec73f-ae53-43c2-801b-258d618fa465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:40.515371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-b5b5f48d-1c9a-4969-8134-fe074f18a36a None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:40.516743 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b5b5f48d-1c9a-4969-8134-fe074f18a36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:40.529445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-723ec73f-ae53-43c2-801b-258d618fa465 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:40.530743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-723ec73f-ae53-43c2-801b-258d618fa465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:40.584003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 425/835] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:34:40.607003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5f48d-1c9a-4969-8134-fe074f18a36a None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:40.610718 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 411/836] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 09:34:40.632951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ec73f-ae53-43c2-801b-258d618fa465 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:40.715539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5f48d-1c9a-4969-8134-fe074f18a36a None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:40.744039 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ec73f-ae53-43c2-801b-258d618fa465 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:40.756076 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bc81a-169e-49b4-90eb-14ac6333dc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:40.757425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bc81a-169e-49b4-90eb-14ac6333dc69 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:40.757990 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bc81a-169e-49b4-90eb-14ac6333dc69 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:40.758595 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bc81a-169e-49b4-90eb-14ac6333dc69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:40.771282 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bc81a-169e-49b4-90eb-14ac6333dc69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:40.780872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-250bc81a-169e-49b4-90eb-14ac6333dc69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:34:40.780872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:40.780872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:40.780872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:40.780872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:40.780872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:40.780872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:40.780872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:40.780872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:40.780872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:40.780872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:40.780872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:40.780872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:40.780872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:40.780872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:40.780872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:40.782045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:40.782045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:40.782045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:40.782045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:40.782045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:40.782045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:40.782045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:40.782045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:40.782045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:40.782045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:40.782045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:40.782045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:40.782045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:40.782045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:40.782045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:40.782045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:40.783107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:40.783107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:40.783107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:40.783107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:40.783107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:40.783107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:40.783107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:40.783107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:40.783107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:40.783107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:40.783107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:34:40.783107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:40.783923 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526e637-b8e0-4e85-84ca-f1d1c4c8345a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:40.784360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526e637-b8e0-4e85-84ca-f1d1c4c8345a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:40.784738 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526e637-b8e0-4e85-84ca-f1d1c4c8345a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:40.785086 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526e637-b8e0-4e85-84ca-f1d1c4c8345a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:40.785834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526e637-b8e0-4e85-84ca-f1d1c4c8345a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:40.787281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526e637-b8e0-4e85-84ca-f1d1c4c8345a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:40.787607 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 426/837] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:34:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:40.793017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bc81a-169e-49b4-90eb-14ac6333dc69 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:40.811031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-c526e637-b8e0-4e85-84ca-f1d1c4c8345a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 09:34:40.811031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:40.811031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:34:40.811031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:34:40.811031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:34:40.811031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:34:40.811031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:34:40.811031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:40.811031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 09:34:40.811031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:40.811031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:34:40.811031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:40.811031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:40.811031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:40.811031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:40.811031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:40.811031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:40.812134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:40.812134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:40.812134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:40.812134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:40.812134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:40.812134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:40.812134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:34:40.812134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:34:40.812134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:34:40.812134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:34:40.812134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:40.812134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:40.812134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:40.812134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:40.812134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:40.812134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:40.813042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:40.813042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:40.813042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:40.813042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:40.813042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:40.813042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:40.813042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:40.813042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:40.813042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:40.813042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:40.813042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:34:40.813042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:40.813042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 09:34:40.813042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:40.821208 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 412/838] 10.222.0.26 () {56 vars in 1114 bytes} [Mon May 9 09:34:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:40.821812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526e637-b8e0-4e85-84ca-f1d1c4c8345a None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:40.915335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bc81a-169e-49b4-90eb-14ac6333dc69 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:40.939759 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526e637-b8e0-4e85-84ca-f1d1c4c8345a None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:40.961170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429286f-bd60-48b0-b884-1a759cf84dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:40.962248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429286f-bd60-48b0-b884-1a759cf84dbd None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:40.962727 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429286f-bd60-48b0-b884-1a759cf84dbd None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:40.963192 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429286f-bd60-48b0-b884-1a759cf84dbd None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:40.964131 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429286f-bd60-48b0-b884-1a759cf84dbd None admin] RBAC: Adding query filter params (name=admin) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:40.964691 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429286f-bd60-48b0-b884-1a759cf84dbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:40.966261 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429286f-bd60-48b0-b884-1a759cf84dbd None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:40.973095 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e19f4-6e2f-42e2-8857-e61589948375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:40.973756 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e19f4-6e2f-42e2-8857-e61589948375 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:40.974031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e19f4-6e2f-42e2-8857-e61589948375 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:40.974303 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e19f4-6e2f-42e2-8857-e61589948375 None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:40.975006 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489e19f4-6e2f-42e2-8857-e61589948375 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:40.975306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489e19f4-6e2f-42e2-8857-e61589948375 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:40.976679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489e19f4-6e2f-42e2-8857-e61589948375 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:40.976895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 427/839] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:34:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:40.982540 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429286f-bd60-48b0-b884-1a759cf84dbd None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:40.984739 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 413/840] 10.222.0.26 () {56 vars in 1114 bytes} [Mon May 9 09:34:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:40.987230 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e19f4-6e2f-42e2-8857-e61589948375 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:41.090705 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429286f-bd60-48b0-b884-1a759cf84dbd None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:41.104760 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e19f4-6e2f-42e2-8857-e61589948375 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:41.130875 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06758ba2-6768-4a49-88eb-a5be0e1217e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:41.131996 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06758ba2-6768-4a49-88eb-a5be0e1217e6 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:41.132490 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06758ba2-6768-4a49-88eb-a5be0e1217e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:41.132983 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06758ba2-6768-4a49-88eb-a5be0e1217e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:41.136993 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405914b-fc1c-4d89-9f81-78fc43f66ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:41.137649 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405914b-fc1c-4d89-9f81-78fc43f66ade None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:41.137926 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405914b-fc1c-4d89-9f81-78fc43f66ade None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:41.138194 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405914b-fc1c-4d89-9f81-78fc43f66ade None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:41.138896 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405914b-fc1c-4d89-9f81-78fc43f66ade None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:41.139198 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405914b-fc1c-4d89-9f81-78fc43f66ade None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:41.140566 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405914b-fc1c-4d89-9f81-78fc43f66ade None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:41.145138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06758ba2-6768-4a49-88eb-a5be0e1217e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:41.146309 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 414/841] 10.222.0.26 () {56 vars in 1114 bytes} [Mon May 9 09:34:40 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:41.146921 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06758ba2-6768-4a49-88eb-a5be0e1217e6 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:41.147542 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239cbf6-36a0-4d7f-8240-6492ef68cb66 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:41.147845 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239cbf6-36a0-4d7f-8240-6492ef68cb66 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:41.148098 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239cbf6-36a0-4d7f-8240-6492ef68cb66 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:41.149740 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 415/842] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:41.158640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2b208-f1ea-4b76-b7b4-104914d330d1 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:41.158990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2b208-f1ea-4b76-b7b4-104914d330d1 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:41.159658 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2b208-f1ea-4b76-b7b4-104914d330d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:41.176264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-06758ba2-6768-4a49-88eb-a5be0e1217e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:41.176264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:41.176264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:41.176264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:41.176264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:41.176264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:41.176264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:41.176264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:41.176264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:41.176264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:41.176264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:41.176264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:41.176264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:41.176264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:41.176264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:41.177459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:41.177459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:41.177459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:41.177459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:41.177459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:41.177459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:41.177459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:41.177459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:41.177459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:41.177459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:41.177459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:41.177459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:41.177459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:41.177459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:41.177459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:41.177459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:41.177459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:41.178591 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:41.178591 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:41.178591 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:41.178591 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:41.180052 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-cba2b208-f1ea-4b76-b7b4-104914d330d1 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:41.185290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-cba2b208-f1ea-4b76-b7b4-104914d330d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:41.185481 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 428/843] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:41.187268 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06758ba2-6768-4a49-88eb-a5be0e1217e6 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:41.277234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 416/844] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:34:41.294471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2b208-f1ea-4b76-b7b4-104914d330d1 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:41.312493 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06758ba2-6768-4a49-88eb-a5be0e1217e6 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:41.358020 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240703d-6cc6-407c-a694-76c827279a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:41.358452 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240703d-6cc6-407c-a694-76c827279a4e None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:41.358734 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240703d-6cc6-407c-a694-76c827279a4e None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:41.359007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240703d-6cc6-407c-a694-76c827279a4e None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:41.359736 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240703d-6cc6-407c-a694-76c827279a4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:41.360045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240703d-6cc6-407c-a694-76c827279a4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:41.362955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240703d-6cc6-407c-a694-76c827279a4e None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:41.376048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 429/845] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:41.383381 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240703d-6cc6-407c-a694-76c827279a4e None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:41.408261 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2b208-f1ea-4b76-b7b4-104914d330d1 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:41.442024 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb6a15-93e6-4feb-b5ba-4185bcb1f815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:41.443029 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb6a15-93e6-4feb-b5ba-4185bcb1f815 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:41.443400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb6a15-93e6-4feb-b5ba-4185bcb1f815 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:41.443797 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb6a15-93e6-4feb-b5ba-4185bcb1f815 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:41.444544 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb6a15-93e6-4feb-b5ba-4185bcb1f815 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:41.446160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb6a15-93e6-4feb-b5ba-4185bcb1f815 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:41.458728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-2bdb6a15-93e6-4feb-b5ba-4185bcb1f815 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 09:34:41.458728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:41.458728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:34:41.458728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:34:41.458728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:34:41.458728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:34:41.458728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:34:41.458728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:41.458728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 09:34:41.458728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:41.458728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:34:41.458728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:41.458728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:41.458728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:41.458728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:41.458728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:41.459843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:41.459843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:41.459843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:41.459843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:41.459843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:41.459843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:41.459843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:41.459843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:34:41.459843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:34:41.459843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:34:41.459843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:34:41.459843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:41.459843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:41.459843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:41.459843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:41.459843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:41.460703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:41.460703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:41.460703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:41.460703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:41.460703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:41.460703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:41.460703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:41.460703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:41.460703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:41.460703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:41.460703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:41.460703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:34:41.460703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:41.460703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 09:34:41.460703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:41.465566 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 417/846] 10.222.0.26 () {56 vars in 1132 bytes} [Mon May 9 09:34:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:41.466720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb6a15-93e6-4feb-b5ba-4185bcb1f815 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:41.485742 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240703d-6cc6-407c-a694-76c827279a4e None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:41.519640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa5544-2396-456b-94be-e35624f494de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:41.521298 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa5544-2396-456b-94be-e35624f494de None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:41.521781 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa5544-2396-456b-94be-e35624f494de None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:41.522260 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa5544-2396-456b-94be-e35624f494de None admin] PATH_INFO: `/v3/users/swift` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:41.529866 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa5544-2396-456b-94be-e35624f494de None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:41.531565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa5544-2396-456b-94be-e35624f494de None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:41.553182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-c1fa5544-2396-456b-94be-e35624f494de None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 09:34:41.553182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:41.553182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:41.553182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:41.553182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:41.553182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:41.553182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:41.553182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:41.553182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:41.553182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:41.553182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:41.553182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:41.553182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:41.553182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:41.553182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:41.553182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:41.554209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:41.554209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:41.554209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:41.554209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:41.554209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:41.554209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:41.554209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:41.554209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:41.554209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:41.554209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:41.554209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:41.554209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:41.554209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:41.554209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:41.554209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:41.554209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:41.554209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:41.554209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:41.555197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:41.555197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:41.555197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:41.555197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:41.555197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:41.555197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:41.555197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:41.555197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:41.555197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 09:34:41.555197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:41.564172 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 430/847] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:34:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:41.568963 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb6a15-93e6-4feb-b5ba-4185bcb1f815 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:41.570531 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa5544-2396-456b-94be-e35624f494de None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:41.606572 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9f190-86d0-410f-a58b-3391c2b8d785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:41.607242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9f190-86d0-410f-a58b-3391c2b8d785 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:41.607518 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9f190-86d0-410f-a58b-3391c2b8d785 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:41.607786 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9f190-86d0-410f-a58b-3391c2b8d785 None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:41.608551 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e9f190-86d0-410f-a58b-3391c2b8d785 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:41.608828 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e9f190-86d0-410f-a58b-3391c2b8d785 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:41.610279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e9f190-86d0-410f-a58b-3391c2b8d785 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:41.619298 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 418/848] 10.222.0.26 () {56 vars in 1126 bytes} [Mon May 9 09:34:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:41.620444 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9f190-86d0-410f-a58b-3391c2b8d785 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:41.689529 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa5544-2396-456b-94be-e35624f494de None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:41.721976 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9f190-86d0-410f-a58b-3391c2b8d785 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:41.741803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c0b17-ad5d-46b0-b61e-bc7ff478b6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:41.742639 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c0b17-ad5d-46b0-b61e-bc7ff478b6e8 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:41.743280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c0b17-ad5d-46b0-b61e-bc7ff478b6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:41.743552 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c0b17-ad5d-46b0-b61e-bc7ff478b6e8 None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:41.744407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c0b17-ad5d-46b0-b61e-bc7ff478b6e8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:41.744744 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c0b17-ad5d-46b0-b61e-bc7ff478b6e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:41.746958 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c0b17-ad5d-46b0-b61e-bc7ff478b6e8 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:41.759966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb19f13-1c93-410a-9c64-2244d7fc1184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:41.760631 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb19f13-1c93-410a-9c64-2244d7fc1184 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:41.760939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb19f13-1c93-410a-9c64-2244d7fc1184 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:41.761208 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb19f13-1c93-410a-9c64-2244d7fc1184 None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:41.761912 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb19f13-1c93-410a-9c64-2244d7fc1184 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:41.762212 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb19f13-1c93-410a-9c64-2244d7fc1184 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:41.763573 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb19f13-1c93-410a-9c64-2244d7fc1184 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:41.771703 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 431/849] 10.222.0.26 () {56 vars in 1135 bytes} [Mon May 9 09:34:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:41.771879 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 419/850] 10.222.0.26 () {56 vars in 1126 bytes} [Mon May 9 09:34:41 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:41.772931 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb19f13-1c93-410a-9c64-2244d7fc1184 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:41.773624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07735c-3810-460f-ae7e-e0883b25083f None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:41.774261 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07735c-3810-460f-ae7e-e0883b25083f None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:41.774732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07735c-3810-460f-ae7e-e0883b25083f None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:41.776710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 432/851] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:41.782598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cdd5a-bc95-443d-980d-ff2faf5b9992 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:41.783210 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cdd5a-bc95-443d-980d-ff2faf5b9992 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:41.783689 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cdd5a-bc95-443d-980d-ff2faf5b9992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:41.801243 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-0c5cdd5a-bc95-443d-980d-ff2faf5b9992 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:41.802582 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-0c5cdd5a-bc95-443d-980d-ff2faf5b9992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:41.877155 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 433/852] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 09:34:41.880826 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb19f13-1c93-410a-9c64-2244d7fc1184 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:41.895292 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cdd5a-bc95-443d-980d-ff2faf5b9992 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:41.920421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d048b8-2a9e-4456-990e-63930bdc9462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:41.921539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d048b8-2a9e-4456-990e-63930bdc9462 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:41.922065 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d048b8-2a9e-4456-990e-63930bdc9462 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:41.922554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d048b8-2a9e-4456-990e-63930bdc9462 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:41.994417 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cdd5a-bc95-443d-980d-ff2faf5b9992 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:41.997638 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d048b8-2a9e-4456-990e-63930bdc9462 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:41.999178 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d048b8-2a9e-4456-990e-63930bdc9462 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:42.029138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09791578-cb2d-4c97-a1bb-21058670760e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:42.029940 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09791578-cb2d-4c97-a1bb-21058670760e None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:42.030204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09791578-cb2d-4c97-a1bb-21058670760e None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:42.030475 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09791578-cb2d-4c97-a1bb-21058670760e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:42.040752 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09791578-cb2d-4c97-a1bb-21058670760e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:42.048539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-09791578-cb2d-4c97-a1bb-21058670760e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:42.048539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:42.048539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:42.048539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:42.048539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:42.048539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:42.048539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:42.048539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:42.048539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:42.048539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:42.048539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:42.048539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:42.048539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:42.048539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:42.048539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:42.048539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:42.049547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:42.049547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:42.049547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:42.049547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:42.049547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:42.049547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:42.049547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:42.049547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:42.049547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:42.049547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:42.049547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:42.049547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:42.049547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:42.049547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:42.049547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:42.049547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:42.050390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:42.050390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:42.050390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:42.050390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:42.050390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:42.050390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:42.050390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:42.050390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:42.050390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:42.050390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:42.050390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 09:34:42.050390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:42.050390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 434/853] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:34:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:42.055881 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09791578-cb2d-4c97-a1bb-21058670760e None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:42.121978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 420/854] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:34:41 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 09:34:42.127353 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d048b8-2a9e-4456-990e-63930bdc9462 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:42.176624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09791578-cb2d-4c97-a1bb-21058670760e None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:42.217766 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a2765-8059-4d53-8d53-27a4fe8e4004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:42.219356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a2765-8059-4d53-8d53-27a4fe8e4004 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:42.219838 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a2765-8059-4d53-8d53-27a4fe8e4004 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:42.220455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a2765-8059-4d53-8d53-27a4fe8e4004 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:42.221991 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a2765-8059-4d53-8d53-27a4fe8e4004 None admin] RBAC: Adding query filter params (name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:42.222521 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a2765-8059-4d53-8d53-27a4fe8e4004 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:42.224801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a2765-8059-4d53-8d53-27a4fe8e4004 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:42.226292 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d048b8-2a9e-4456-990e-63930bdc9462 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:42.238780 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 435/855] 10.222.0.26 () {56 vars in 1103 bytes} [Mon May 9 09:34:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:42.239604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a2765-8059-4d53-8d53-27a4fe8e4004 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:42.267662 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200610d-7363-4479-919f-097ff88bfd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:42.268880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200610d-7363-4479-919f-097ff88bfd65 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:42.269451 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200610d-7363-4479-919f-097ff88bfd65 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:42.269884 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200610d-7363-4479-919f-097ff88bfd65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:42.283242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200610d-7363-4479-919f-097ff88bfd65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:42.285475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200610d-7363-4479-919f-097ff88bfd65 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:42.307787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-e200610d-7363-4479-919f-097ff88bfd65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:42.307787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:42.307787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:42.307787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:42.307787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:42.307787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:42.307787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:42.307787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:42.307787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:42.307787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:42.307787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:42.307787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:42.307787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:42.307787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:42.307787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:42.309107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:42.309107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:42.309107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:42.309107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:42.309107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:42.309107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:42.309107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:42.309107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:42.309107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:42.309107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:42.309107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:42.309107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:42.309107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:42.309107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:42.309107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:42.309107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:42.309107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:42.310233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:42.310233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:42.310233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:42.310233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:42.312940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 421/856] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:42.319244 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200610d-7363-4479-919f-097ff88bfd65 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:42.350975 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a2765-8059-4d53-8d53-27a4fe8e4004 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:42.392411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3a1d2-63a9-43ac-91ad-6938b8cd7e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:42.393174 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3a1d2-63a9-43ac-91ad-6938b8cd7e83 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:42.393449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3a1d2-63a9-43ac-91ad-6938b8cd7e83 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:42.393711 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3a1d2-63a9-43ac-91ad-6938b8cd7e83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:42.407976 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3a1d2-63a9-43ac-91ad-6938b8cd7e83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:42.409830 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3a1d2-63a9-43ac-91ad-6938b8cd7e83 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:42.431377 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200610d-7363-4479-919f-097ff88bfd65 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:42.431708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-24c3a1d2-63a9-43ac-91ad-6938b8cd7e83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:42.431708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:42.431708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:42.431708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:42.431708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:42.431708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:42.431708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:42.431708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:42.431708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:42.431708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:42.431708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:42.431708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:42.431708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:42.431708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:42.431708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:42.432574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:42.432574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:42.432574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:42.432574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:42.432574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:42.432574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:42.432574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:42.432574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:42.432574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:42.432574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:42.432574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:42.432574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:42.432574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:42.432574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:42.432574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:42.432574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:42.432574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:42.433523 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:42.433523 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:42.433523 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:42.433523 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:42.436750 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 436/857] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:42.439937 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3a1d2-63a9-43ac-91ad-6938b8cd7e83 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:42.472273 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344227b6-da39-4de5-a44e-3afca8f2c378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:42.473093 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344227b6-da39-4de5-a44e-3afca8f2c378 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:42.473483 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344227b6-da39-4de5-a44e-3afca8f2c378 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:42.473775 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344227b6-da39-4de5-a44e-3afca8f2c378 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:42.474496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344227b6-da39-4de5-a44e-3afca8f2c378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:42.474795 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344227b6-da39-4de5-a44e-3afca8f2c378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:42.476777 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344227b6-da39-4de5-a44e-3afca8f2c378 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:42.492076 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 422/858] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:42.499622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344227b6-da39-4de5-a44e-3afca8f2c378 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:42.557145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3a1d2-63a9-43ac-91ad-6938b8cd7e83 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:42.598407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023e587-de42-4ee1-b927-542c4c531e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:42.599411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023e587-de42-4ee1-b927-542c4c531e36 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:42.599901 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023e587-de42-4ee1-b927-542c4c531e36 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:42.600362 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023e587-de42-4ee1-b927-542c4c531e36 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:42.601318 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f023e587-de42-4ee1-b927-542c4c531e36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:42.601826 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f023e587-de42-4ee1-b927-542c4c531e36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:42.603418 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f023e587-de42-4ee1-b927-542c4c531e36 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:42.614659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344227b6-da39-4de5-a44e-3afca8f2c378 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:42.616099 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 437/859] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:42.624440 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023e587-de42-4ee1-b927-542c4c531e36 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:42.651526 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeaad71-991b-495d-ad77-3e1b1afb5c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:42.652648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeaad71-991b-495d-ad77-3e1b1afb5c90 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:42.653155 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeaad71-991b-495d-ad77-3e1b1afb5c90 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:42.653615 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeaad71-991b-495d-ad77-3e1b1afb5c90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:42.660525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeaad71-991b-495d-ad77-3e1b1afb5c90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:42.662237 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeaad71-991b-495d-ad77-3e1b1afb5c90 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:42.676885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-7aeaad71-991b-495d-ad77-3e1b1afb5c90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:42.676885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:42.676885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:42.676885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:42.676885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:42.676885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:42.676885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:42.676885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:42.676885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:42.676885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:42.676885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:42.676885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:42.676885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:42.676885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:42.676885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:42.677819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:42.677819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:42.677819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:42.677819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:42.677819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:42.677819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:42.677819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:42.677819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:42.677819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:42.677819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:42.677819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:42.677819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:42.677819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:42.677819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:42.677819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:42.677819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:42.677819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:42.678729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:42.678729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:42.678729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:42.678729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:42.678729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:42.678729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:42.678729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:42.678729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:42.684924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 423/860] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:42.687216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeaad71-991b-495d-ad77-3e1b1afb5c90 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:42.737398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023e587-de42-4ee1-b927-542c4c531e36 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:42.780751 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5bef28-5d5f-4b80-9931-466a83b50369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:42.781833 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bef28-5d5f-4b80-9931-466a83b50369 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:42.782110 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bef28-5d5f-4b80-9931-466a83b50369 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:42.782565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bef28-5d5f-4b80-9931-466a83b50369 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:42.791427 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5bef28-5d5f-4b80-9931-466a83b50369 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:42.793351 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5bef28-5d5f-4b80-9931-466a83b50369 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:42.801140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeaad71-991b-495d-ad77-3e1b1afb5c90 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:42.810532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-6f5bef28-5d5f-4b80-9931-466a83b50369 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 09:34:42.810532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:42.810532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:42.810532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:42.810532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:42.810532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:42.810532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:42.810532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:42.810532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:42.810532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:42.810532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:42.810532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:42.810532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:42.810532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:42.810532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:42.810532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:42.811565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:42.811565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:42.811565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:42.811565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:42.811565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:42.811565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:42.811565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:42.811565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:42.811565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:42.811565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:42.811565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:42.811565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:42.811565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:42.811565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:42.811565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:42.811565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:42.811565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:42.811565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:42.812638 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:42.812638 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:42.812638 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:42.812638 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:42.812638 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:42.812638 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:42.812638 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:42.812638 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:42.812638 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 09:34:42.812638 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:42.816715 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 438/861] 10.222.0.26 () {56 vars in 1117 bytes} [Mon May 9 09:34:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:42.817962 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5bef28-5d5f-4b80-9931-466a83b50369 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:42.843999 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0f97c-f2ff-43a5-a2d4-4b2e9231070f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:42.845081 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0f97c-f2ff-43a5-a2d4-4b2e9231070f None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:42.845592 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0f97c-f2ff-43a5-a2d4-4b2e9231070f None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:42.846061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0f97c-f2ff-43a5-a2d4-4b2e9231070f None admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:42.847085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0f97c-f2ff-43a5-a2d4-4b2e9231070f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:42.847607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0f97c-f2ff-43a5-a2d4-4b2e9231070f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:42.849402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0f97c-f2ff-43a5-a2d4-4b2e9231070f None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:42.866725 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 424/862] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:42.867947 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0f97c-f2ff-43a5-a2d4-4b2e9231070f None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:42.915629 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5bef28-5d5f-4b80-9931-466a83b50369 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:42.948588 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0515d1f-3990-4056-a1d0-f980c2f4b589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:42.949221 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0515d1f-3990-4056-a1d0-f980c2f4b589 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:42.949488 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0515d1f-3990-4056-a1d0-f980c2f4b589 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:42.949757 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0515d1f-3990-4056-a1d0-f980c2f4b589 None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:42.950625 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0515d1f-3990-4056-a1d0-f980c2f4b589 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:42.950927 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0515d1f-3990-4056-a1d0-f980c2f4b589 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:42.952414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0515d1f-3990-4056-a1d0-f980c2f4b589 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:42.972647 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0f97c-f2ff-43a5-a2d4-4b2e9231070f None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:42.975846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 439/863] 10.222.0.26 () {56 vars in 1149 bytes} [Mon May 9 09:34:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:42.976694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0515d1f-3990-4056-a1d0-f980c2f4b589 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:43.008258 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc0fcf-5b86-4138-b379-ea1981a6076e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:43.009427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc0fcf-5b86-4138-b379-ea1981a6076e None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:43.009930 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc0fcf-5b86-4138-b379-ea1981a6076e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:43.010401 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc0fcf-5b86-4138-b379-ea1981a6076e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:43.011434 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc0fcf-5b86-4138-b379-ea1981a6076e None admin] RBAC: Adding query filter params (role.id=70ac549382b14b2cbb8b35e5dfc648d4, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=380e04358f2349aa912d6a97518877cd) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:43.012010 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc0fcf-5b86-4138-b379-ea1981a6076e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:43.013679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc0fcf-5b86-4138-b379-ea1981a6076e None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:43.026920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 425/864] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:34:42 2022] GET /identity/v3/role_assignments?user.id=380e04358f2349aa912d6a97518877cd&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=70ac549382b14b2cbb8b35e5dfc648d4 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:43.079573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0515d1f-3990-4056-a1d0-f980c2f4b589 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:43.120020 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e0b6f-39dd-4cb1-a459-12260a114d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:43.121030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e0b6f-39dd-4cb1-a459-12260a114d4d None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:43.121304 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e0b6f-39dd-4cb1-a459-12260a114d4d None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:43.121572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e0b6f-39dd-4cb1-a459-12260a114d4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:43.191978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e0b6f-39dd-4cb1-a459-12260a114d4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:43.193390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e0b6f-39dd-4cb1-a459-12260a114d4d None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:43.295319 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 440/865] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:34:42 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 09:34:43.301153 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc0fcf-5b86-4138-b379-ea1981a6076e None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:43.404825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc0fcf-5b86-4138-b379-ea1981a6076e None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:43.437670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf00dc3-983f-4ba9-9cf9-6ab96b1257e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:43.438457 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf00dc3-983f-4ba9-9cf9-6ab96b1257e7 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:43.438730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf00dc3-983f-4ba9-9cf9-6ab96b1257e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:43.438995 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf00dc3-983f-4ba9-9cf9-6ab96b1257e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:43.449527 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf00dc3-983f-4ba9-9cf9-6ab96b1257e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:43.450905 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf00dc3-983f-4ba9-9cf9-6ab96b1257e7 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:43.467276 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bbba1-3658-4e9a-94cc-41dcda50bc2e None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:43.467599 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bbba1-3658-4e9a-94cc-41dcda50bc2e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:43.467877 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bbba1-3658-4e9a-94cc-41dcda50bc2e None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:43.468205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-caf00dc3-983f-4ba9-9cf9-6ab96b1257e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:43.468205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:43.468205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:43.468205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:43.468205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:43.468205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:43.468205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:43.468205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:43.468205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:43.468205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:43.468205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:43.468205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:43.468205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:43.468205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:43.468205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:43.469236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:43.469236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:43.469236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:43.469236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:43.469236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:43.469236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:43.469236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:43.469236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:43.469236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:43.469236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:43.469236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:43.469236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:43.469236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:43.469236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:43.469236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:43.469236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:43.469236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:43.470321 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:43.470321 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:43.470321 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:43.470321 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:43.470562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 441/866] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:34:43.473533 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 426/867] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:43.475062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222dead2-8a60-450c-b445-95298c6039e3 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:43.475305 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf00dc3-983f-4ba9-9cf9-6ab96b1257e7 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:43.475476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222dead2-8a60-450c-b445-95298c6039e3 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:43.475759 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222dead2-8a60-450c-b445-95298c6039e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:43.491989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-222dead2-8a60-450c-b445-95298c6039e3 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:43.493307 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-222dead2-8a60-450c-b445-95298c6039e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:43.568756 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 442/868] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 09:34:43.589973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222dead2-8a60-450c-b445-95298c6039e3 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:43.601447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf00dc3-983f-4ba9-9cf9-6ab96b1257e7 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:43.637946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777fef1-f0c4-4640-8c1a-c4bcea44769c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:43.638602 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777fef1-f0c4-4640-8c1a-c4bcea44769c None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:43.638870 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777fef1-f0c4-4640-8c1a-c4bcea44769c None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:43.639143 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777fef1-f0c4-4640-8c1a-c4bcea44769c None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:43.639878 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777fef1-f0c4-4640-8c1a-c4bcea44769c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:43.640180 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777fef1-f0c4-4640-8c1a-c4bcea44769c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:43.641797 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777fef1-f0c4-4640-8c1a-c4bcea44769c None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:43.654471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 427/869] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:43.659547 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777fef1-f0c4-4640-8c1a-c4bcea44769c None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:43.712713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222dead2-8a60-450c-b445-95298c6039e3 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:43.746017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d4fb4-819a-4ce8-80fe-4a6f61417296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:43.746675 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d4fb4-819a-4ce8-80fe-4a6f61417296 None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:43.746941 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d4fb4-819a-4ce8-80fe-4a6f61417296 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:43.747209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d4fb4-819a-4ce8-80fe-4a6f61417296 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:43.747938 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d4fb4-819a-4ce8-80fe-4a6f61417296 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:43.749328 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d4fb4-819a-4ce8-80fe-4a6f61417296 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:43.763974 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777fef1-f0c4-4640-8c1a-c4bcea44769c None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:43.768697 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 443/870] 10.222.0.26 () {60 vars in 1141 bytes} [Mon May 9 09:34:43 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:34:43.815729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74763e8-5b85-42a2-a75f-14678f2564bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:43.816623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74763e8-5b85-42a2-a75f-14678f2564bf None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:43.816925 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74763e8-5b85-42a2-a75f-14678f2564bf None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:43.817192 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74763e8-5b85-42a2-a75f-14678f2564bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:43.825170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74763e8-5b85-42a2-a75f-14678f2564bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:43.826708 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74763e8-5b85-42a2-a75f-14678f2564bf None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:43.842105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-a74763e8-5b85-42a2-a75f-14678f2564bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:43.842105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:43.842105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:43.842105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:43.842105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:43.842105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:43.842105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:43.842105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:43.842105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:43.842105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:43.842105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:43.842105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:43.842105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:43.842105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:43.842105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:43.843084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:43.843084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:43.843084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:43.843084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:43.843084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:43.843084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:43.843084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:43.843084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:43.843084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:43.843084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:43.843084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:43.843084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:43.843084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:43.843084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:43.843084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:43.843084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:43.843084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:43.843998 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:43.843998 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:43.843998 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:43.843998 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:43.843998 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:43.843998 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:43.843998 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:43.843998 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:43.848267 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 428/871] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:43.849716 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d4fb4-819a-4ce8-80fe-4a6f61417296 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:43.962921 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d4fb4-819a-4ce8-80fe-4a6f61417296 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:43.995611 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b78c7-ce83-4a1a-9498-862c14f0f7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:43.996271 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b78c7-ce83-4a1a-9498-862c14f0f7fb None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:43.996543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b78c7-ce83-4a1a-9498-862c14f0f7fb None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:43.996838 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b78c7-ce83-4a1a-9498-862c14f0f7fb None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:43.997657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b78c7-ce83-4a1a-9498-862c14f0f7fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:43.998008 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b78c7-ce83-4a1a-9498-862c14f0f7fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:43.999484 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b78c7-ce83-4a1a-9498-862c14f0f7fb None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:44.012792 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 444/872] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:44.017845 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74763e8-5b85-42a2-a75f-14678f2564bf None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:44.087264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2ef60-4aea-455f-8b0a-cac816d8a72f None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:44.087838 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2ef60-4aea-455f-8b0a-cac816d8a72f None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:44.088117 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2ef60-4aea-455f-8b0a-cac816d8a72f None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:44.090048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 445/873] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:44.095110 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d23fc3-56aa-4dd7-84a9-91822bfd2113 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:44.095388 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d23fc3-56aa-4dd7-84a9-91822bfd2113 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:44.095654 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d23fc3-56aa-4dd7-84a9-91822bfd2113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:44.111921 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-40d23fc3-56aa-4dd7-84a9-91822bfd2113 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:44.113158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-40d23fc3-56aa-4dd7-84a9-91822bfd2113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:44.128883 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74763e8-5b85-42a2-a75f-14678f2564bf None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:44.161551 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbbdd84-37fd-4cfc-a2fd-0b8edea33abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:44.162205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbdd84-37fd-4cfc-a2fd-0b8edea33abc None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:44.162483 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbdd84-37fd-4cfc-a2fd-0b8edea33abc None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:44.162752 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbdd84-37fd-4cfc-a2fd-0b8edea33abc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:44.163581 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbbdd84-37fd-4cfc-a2fd-0b8edea33abc None admin] RBAC: Adding query filter params (role.id=f55c3728df78466989d0cd1b13c5ea28, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=af62b2d3f73f482b8db4173af33752c6) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:44.163887 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbbdd84-37fd-4cfc-a2fd-0b8edea33abc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:44.165371 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbbdd84-37fd-4cfc-a2fd-0b8edea33abc None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:44.171419 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 429/874] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:34:44 2022] GET /identity/v3/role_assignments?user.id=af62b2d3f73f482b8db4173af33752c6&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=f55c3728df78466989d0cd1b13c5ea28 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:44.191741 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 446/875] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:44 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:34:44.214151 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbbdd84-37fd-4cfc-a2fd-0b8edea33abc None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:44.323899 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbbdd84-37fd-4cfc-a2fd-0b8edea33abc None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:44.360175 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5d690-6c9e-4782-90e3-95355d86786c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:44.361283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5d690-6c9e-4782-90e3-95355d86786c None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:44.361756 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5d690-6c9e-4782-90e3-95355d86786c None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:44.362215 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5d690-6c9e-4782-90e3-95355d86786c None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:44.363139 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba5d690-6c9e-4782-90e3-95355d86786c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:44.364779 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba5d690-6c9e-4782-90e3-95355d86786c None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:44.384781 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 430/876] 10.222.0.26 () {60 vars in 1141 bytes} [Mon May 9 09:34:44 2022] POST /identity/v3/services => generated 269 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:34:45.373644 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f6197-48d8-4587-80b1-48d6d76a93c5 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:45.374214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f6197-48d8-4587-80b1-48d6d76a93c5 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:45.374490 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f6197-48d8-4587-80b1-48d6d76a93c5 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:45.376528 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 447/877] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:45.382116 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce7baf-5120-4ab9-8311-1c9e0fe19778 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:45.382946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce7baf-5120-4ab9-8311-1c9e0fe19778 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:45.383492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce7baf-5120-4ab9-8311-1c9e0fe19778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:45.401391 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-e3ce7baf-5120-4ab9-8311-1c9e0fe19778 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:45.402991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e3ce7baf-5120-4ab9-8311-1c9e0fe19778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:45.482738 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 431/878] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:34:45.505511 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f6197-48d8-4587-80b1-48d6d76a93c5 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:45.620521 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f6197-48d8-4587-80b1-48d6d76a93c5 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:45.655209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc078e84-aad8-470d-9e2b-3a69a0b3fc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:45.655945 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc078e84-aad8-470d-9e2b-3a69a0b3fc41 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:45.656226 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc078e84-aad8-470d-9e2b-3a69a0b3fc41 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:45.656497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc078e84-aad8-470d-9e2b-3a69a0b3fc41 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:45.657187 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc078e84-aad8-470d-9e2b-3a69a0b3fc41 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:45.658560 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc078e84-aad8-470d-9e2b-3a69a0b3fc41 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:45.670652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-cc078e84-aad8-470d-9e2b-3a69a0b3fc41 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 09:34:45.670652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:45.670652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:34:45.670652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:34:45.670652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:34:45.670652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:34:45.670652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:34:45.670652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:45.670652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 09:34:45.670652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:45.670652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:34:45.670652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:45.670652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:45.670652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:45.670652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:45.670652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:45.670652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:45.671677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:45.671677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:45.671677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:45.671677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:45.671677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:45.671677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:45.671677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:34:45.671677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:34:45.671677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:34:45.671677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:34:45.671677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:45.671677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:45.671677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:45.671677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:45.671677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:45.671677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:45.672530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:45.672530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:45.672530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:45.672530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:45.672530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:45.672530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:45.672530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:45.672530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:45.672530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:45.672530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:45.672530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:34:45.672530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:45.672530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 09:34:45.672530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:45.672530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 448/879] 10.222.0.26 () {56 vars in 1126 bytes} [Mon May 9 09:34:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:45.680051 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce7baf-5120-4ab9-8311-1c9e0fe19778 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:45.778544 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce7baf-5120-4ab9-8311-1c9e0fe19778 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:45.812771 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9aac4-81d9-4af6-9817-debad9a9b638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:45.813429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9aac4-81d9-4af6-9817-debad9a9b638 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:45.813706 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9aac4-81d9-4af6-9817-debad9a9b638 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:45.813972 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9aac4-81d9-4af6-9817-debad9a9b638 None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:45.814670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9aac4-81d9-4af6-9817-debad9a9b638 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:45.814967 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9aac4-81d9-4af6-9817-debad9a9b638 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:45.816591 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9aac4-81d9-4af6-9817-debad9a9b638 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:45.821542 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 432/880] 10.222.0.26 () {56 vars in 1122 bytes} [Mon May 9 09:34:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:45.829733 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc078e84-aad8-470d-9e2b-3a69a0b3fc41 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:45.935371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc078e84-aad8-470d-9e2b-3a69a0b3fc41 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:45.967925 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7a62f-df13-447a-b39b-1f955651f30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:45.968591 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7a62f-df13-447a-b39b-1f955651f30a None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:45.968894 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7a62f-df13-447a-b39b-1f955651f30a None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:45.969165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7a62f-df13-447a-b39b-1f955651f30a None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:45.969875 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7a62f-df13-447a-b39b-1f955651f30a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:45.970177 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7a62f-df13-447a-b39b-1f955651f30a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:45.971544 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7a62f-df13-447a-b39b-1f955651f30a None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:45.976323 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 449/881] 10.222.0.26 () {56 vars in 1122 bytes} [Mon May 9 09:34:45 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:46.060728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e96cf-02e3-4c17-8639-404649319e52 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:46.061047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e96cf-02e3-4c17-8639-404649319e52 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:46.061278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e96cf-02e3-4c17-8639-404649319e52 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:46.062945 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 433/882] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:46.068748 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f966c-b59f-46bc-a8b6-718a24847170 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:46.069084 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f966c-b59f-46bc-a8b6-718a24847170 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:46.069309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f966c-b59f-46bc-a8b6-718a24847170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:46.086934 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-ea9f966c-b59f-46bc-a8b6-718a24847170 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:46.088207 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-ea9f966c-b59f-46bc-a8b6-718a24847170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:46.162195 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 450/883] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:34:46.186090 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e96cf-02e3-4c17-8639-404649319e52 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:46.288062 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e96cf-02e3-4c17-8639-404649319e52 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:46.325161 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc39031-7aee-410a-b324-ba088171f5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:46.326345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc39031-7aee-410a-b324-ba088171f5bd None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:46.326853 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc39031-7aee-410a-b324-ba088171f5bd None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:46.327322 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc39031-7aee-410a-b324-ba088171f5bd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:46.328185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc39031-7aee-410a-b324-ba088171f5bd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:46.329788 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc39031-7aee-410a-b324-ba088171f5bd None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:46.342209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-ffc39031-7aee-410a-b324-ba088171f5bd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 09:34:46.342209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:46.342209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:34:46.342209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:34:46.342209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:34:46.342209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:34:46.342209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:34:46.342209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:46.342209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 09:34:46.342209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:46.342209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:34:46.342209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:46.342209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:46.342209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:46.342209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:46.342209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:46.342209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:46.343234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:46.343234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:46.343234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:46.343234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:46.343234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:46.343234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:46.343234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:34:46.343234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:34:46.343234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:34:46.343234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:34:46.343234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:46.343234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:46.343234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:46.343234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:46.343234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:46.343234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:46.344091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:46.344091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:46.344091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:46.344091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:46.344091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:46.344091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:46.344091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:46.344091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:46.344091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:46.344091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:46.344091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:34:46.344091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:46.344091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 09:34:46.344091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:46.346946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 434/884] 10.222.0.26 () {56 vars in 1114 bytes} [Mon May 9 09:34:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:46.353191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f966c-b59f-46bc-a8b6-718a24847170 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:46.458477 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f966c-b59f-46bc-a8b6-718a24847170 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:46.492033 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaba2e6-6704-4020-a1e7-38f5cd25e44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:46.492833 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaba2e6-6704-4020-a1e7-38f5cd25e44d None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:46.493116 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaba2e6-6704-4020-a1e7-38f5cd25e44d None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:46.493385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaba2e6-6704-4020-a1e7-38f5cd25e44d None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:46.494151 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaba2e6-6704-4020-a1e7-38f5cd25e44d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:46.494450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaba2e6-6704-4020-a1e7-38f5cd25e44d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:46.496193 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaba2e6-6704-4020-a1e7-38f5cd25e44d None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:46.501187 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 451/885] 10.222.0.26 () {56 vars in 1114 bytes} [Mon May 9 09:34:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:46.506772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc39031-7aee-410a-b324-ba088171f5bd None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:46.615687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42b7fb-d82b-4975-8a37-b3a4680ca06c None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:46.615687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42b7fb-d82b-4975-8a37-b3a4680ca06c None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:46.615687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42b7fb-d82b-4975-8a37-b3a4680ca06c None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:46.615687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 452/886] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:34:46.615687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571bf2e-340a-40e1-92fb-2a016a5a62c7 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:46.616114 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571bf2e-340a-40e1-92fb-2a016a5a62c7 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:46.616114 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571bf2e-340a-40e1-92fb-2a016a5a62c7 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:46.617575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 453/887] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:34:46.618370 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79897173-7f46-4874-abb1-a6f920b8e798 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:46.618645 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79897173-7f46-4874-abb1-a6f920b8e798 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:46.618914 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79897173-7f46-4874-abb1-a6f920b8e798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:46.623345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc39031-7aee-410a-b324-ba088171f5bd None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:46.635551 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-79897173-7f46-4874-abb1-a6f920b8e798 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:46.636772 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-79897173-7f46-4874-abb1-a6f920b8e798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:46.657884 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f7069-65a8-4160-868d-1d2daf69787d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:46.658543 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f7069-65a8-4160-868d-1d2daf69787d None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:46.658832 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f7069-65a8-4160-868d-1d2daf69787d None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:46.659086 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f7069-65a8-4160-868d-1d2daf69787d None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:46.659786 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f7069-65a8-4160-868d-1d2daf69787d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:46.660278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f7069-65a8-4160-868d-1d2daf69787d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:46.661471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f7069-65a8-4160-868d-1d2daf69787d None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:46.666191 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 435/888] 10.222.0.26 () {56 vars in 1114 bytes} [Mon May 9 09:34:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:46.667336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660a66c-1dcc-45fa-aef7-469fc1d3f753 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:46.667634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660a66c-1dcc-45fa-aef7-469fc1d3f753 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:46.667881 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660a66c-1dcc-45fa-aef7-469fc1d3f753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:46.690746 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-6660a66c-1dcc-45fa-aef7-469fc1d3f753 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:46.692415 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-6660a66c-1dcc-45fa-aef7-469fc1d3f753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:46.705813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 454/889] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:34:46.706545 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79897173-7f46-4874-abb1-a6f920b8e798 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:46.772480 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 436/890] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:34:46.776135 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660a66c-1dcc-45fa-aef7-469fc1d3f753 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:46.802974 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79897173-7f46-4874-abb1-a6f920b8e798 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:46.836303 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc75fa-a740-4c29-9fbb-4f0e1b716903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:46.837151 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc75fa-a740-4c29-9fbb-4f0e1b716903 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:46.837426 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc75fa-a740-4c29-9fbb-4f0e1b716903 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:46.837714 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc75fa-a740-4c29-9fbb-4f0e1b716903 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:46.838498 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc75fa-a740-4c29-9fbb-4f0e1b716903 None admin] RBAC: Adding query filter params (interface=public, service_id=b8257dffe88445dc8e5adae474b0f104, region_id=RegionOne) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:46.838806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc75fa-a740-4c29-9fbb-4f0e1b716903 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:46.840192 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc75fa-a740-4c29-9fbb-4f0e1b716903 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:46.845319 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 455/891] 10.222.0.26 () {56 vars in 1251 bytes} [Mon May 9 09:34:46 2022] GET /identity/v3/endpoints?service_id=b8257dffe88445dc8e5adae474b0f104&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:46.846316 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc75fa-a740-4c29-9fbb-4f0e1b716903 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:46.887213 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660a66c-1dcc-45fa-aef7-469fc1d3f753 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:46.924316 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858cc00e-205d-4f83-acb4-e0dc7cedc32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:46.925154 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cc00e-205d-4f83-acb4-e0dc7cedc32f None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:46.925438 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cc00e-205d-4f83-acb4-e0dc7cedc32f None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:46.925702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cc00e-205d-4f83-acb4-e0dc7cedc32f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:46.926355 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858cc00e-205d-4f83-acb4-e0dc7cedc32f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:46.928005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858cc00e-205d-4f83-acb4-e0dc7cedc32f None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:46.939312 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc75fa-a740-4c29-9fbb-4f0e1b716903 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:46.939940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-858cc00e-205d-4f83-acb4-e0dc7cedc32f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 09:34:46.939940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:46.939940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:34:46.939940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:34:46.939940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:34:46.939940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:34:46.939940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:34:46.939940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:46.939940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 09:34:46.939940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:46.939940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:34:46.939940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:46.939940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:46.939940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:46.939940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:46.939940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:46.940910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:46.940910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:46.940910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:46.940910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:46.940910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:46.940910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:46.940910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:46.940910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:34:46.940910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:34:46.940910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:34:46.940910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:34:46.940910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:46.940910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:46.940910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:46.940910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:46.940910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:46.941823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:46.941823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:46.941823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:46.941823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:46.941823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:46.941823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:46.941823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:46.941823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:46.941823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:46.941823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:46.941823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:46.941823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:34:46.941823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:46.941823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 09:34:46.941823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:46.946359 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 437/892] 10.222.0.26 () {56 vars in 1132 bytes} [Mon May 9 09:34:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:46.946989 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858cc00e-205d-4f83-acb4-e0dc7cedc32f None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:46.974110 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489face-b63a-49f6-adb1-8bf967b981dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:46.974911 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489face-b63a-49f6-adb1-8bf967b981dd None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:46.975183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489face-b63a-49f6-adb1-8bf967b981dd None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:46.975449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489face-b63a-49f6-adb1-8bf967b981dd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:46.984790 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2489face-b63a-49f6-adb1-8bf967b981dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:46.992886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-2489face-b63a-49f6-adb1-8bf967b981dd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 09:34:46.992886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:46.992886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:46.992886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:46.992886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:46.992886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:46.992886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:46.992886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:46.992886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:46.992886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:46.992886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:46.992886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:46.992886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:46.992886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:46.992886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:46.992886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:46.993819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:46.993819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:46.993819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:46.993819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:46.993819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:46.993819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:46.993819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:46.993819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:46.993819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:46.993819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:46.993819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:46.993819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:46.993819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:46.993819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:46.993819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:46.993819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:46.994704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:46.994704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:46.994704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:46.994704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:46.994704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:46.994704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:46.994704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:46.994704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:46.994704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:46.994704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:46.994704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 09:34:46.994704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:46.994704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 456/893] 10.222.0.26 () {56 vars in 1120 bytes} [Mon May 9 09:34:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:46.995742 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489face-b63a-49f6-adb1-8bf967b981dd None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:47.049036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858cc00e-205d-4f83-acb4-e0dc7cedc32f None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:47.080973 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b5946-9bb6-41ef-934d-6a0939d884f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:47.081629 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b5946-9bb6-41ef-934d-6a0939d884f2 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:47.081901 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b5946-9bb6-41ef-934d-6a0939d884f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:47.082167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b5946-9bb6-41ef-934d-6a0939d884f2 None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:47.082858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b5946-9bb6-41ef-934d-6a0939d884f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:47.084442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b5946-9bb6-41ef-934d-6a0939d884f2 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:47.089981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 438/894] 10.222.0.26 () {56 vars in 1087 bytes} [Mon May 9 09:34:46 2022] GET /identity/v3/services => generated 1268 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:34:47.090764 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b5946-9bb6-41ef-934d-6a0939d884f2 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:47.107804 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489face-b63a-49f6-adb1-8bf967b981dd None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:47.146158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680116a9-753c-4d92-8e3a-683b0af79364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:47.146840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680116a9-753c-4d92-8e3a-683b0af79364 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:47.147114 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680116a9-753c-4d92-8e3a-683b0af79364 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:47.147380 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680116a9-753c-4d92-8e3a-683b0af79364 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:47.148174 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680116a9-753c-4d92-8e3a-683b0af79364 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:47.148478 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680116a9-753c-4d92-8e3a-683b0af79364 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:47.150462 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680116a9-753c-4d92-8e3a-683b0af79364 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:47.156331 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 457/895] 10.222.0.26 () {56 vars in 1126 bytes} [Mon May 9 09:34:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:47.162220 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680116a9-753c-4d92-8e3a-683b0af79364 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:47.227548 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b5946-9bb6-41ef-934d-6a0939d884f2 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:47.265960 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61c8be-ea98-4ca3-b2ac-6545322ed5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:47.266640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61c8be-ea98-4ca3-b2ac-6545322ed5b4 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:47.266914 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61c8be-ea98-4ca3-b2ac-6545322ed5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:47.267182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61c8be-ea98-4ca3-b2ac-6545322ed5b4 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:47.267938 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b61c8be-ea98-4ca3-b2ac-6545322ed5b4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:47.268236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b61c8be-ea98-4ca3-b2ac-6545322ed5b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:47.269681 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b61c8be-ea98-4ca3-b2ac-6545322ed5b4 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:47.278881 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 439/896] 10.222.0.26 () {56 vars in 1115 bytes} [Mon May 9 09:34:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:47.284543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680116a9-753c-4d92-8e3a-683b0af79364 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:47.285071 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61c8be-ea98-4ca3-b2ac-6545322ed5b4 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:47.320401 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec54892-ed13-4e16-ad92-5bc84bf6f79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:47.321098 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec54892-ed13-4e16-ad92-5bc84bf6f79c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:47.321374 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec54892-ed13-4e16-ad92-5bc84bf6f79c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:47.321641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec54892-ed13-4e16-ad92-5bc84bf6f79c None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:47.322349 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec54892-ed13-4e16-ad92-5bc84bf6f79c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:47.322649 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec54892-ed13-4e16-ad92-5bc84bf6f79c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:47.324021 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec54892-ed13-4e16-ad92-5bc84bf6f79c None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:47.329791 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 458/897] 10.222.0.26 () {56 vars in 1126 bytes} [Mon May 9 09:34:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:47.335610 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec54892-ed13-4e16-ad92-5bc84bf6f79c None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:47.406237 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61c8be-ea98-4ca3-b2ac-6545322ed5b4 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:47.444169 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec54892-ed13-4e16-ad92-5bc84bf6f79c None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:47.447968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147deb41-d763-489e-b121-faf631ecbaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:47.448797 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147deb41-d763-489e-b121-faf631ecbaf9 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:47.449077 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147deb41-d763-489e-b121-faf631ecbaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:47.449344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147deb41-d763-489e-b121-faf631ecbaf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:47.462325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147deb41-d763-489e-b121-faf631ecbaf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:47.463780 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147deb41-d763-489e-b121-faf631ecbaf9 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:47.479125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e367e8-347f-4ac0-8234-e883ba082b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:47.480492 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e367e8-347f-4ac0-8234-e883ba082b9f None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:47.481004 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e367e8-347f-4ac0-8234-e883ba082b9f None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:47.481473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e367e8-347f-4ac0-8234-e883ba082b9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:47.482483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e367e8-347f-4ac0-8234-e883ba082b9f None admin] RBAC: Adding query filter params (interface=public, service_id=c73fb2bd07a1424e9df7d2892097e082, region_id=RegionOne) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:47.483036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e367e8-347f-4ac0-8234-e883ba082b9f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:47.484027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-147deb41-d763-489e-b121-faf631ecbaf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:47.484027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:47.484027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:47.484027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:47.484027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:47.484027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:47.484027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:47.484027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:47.484027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:47.484027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:47.484027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:47.484027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:47.484027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:47.484027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:47.484027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:47.484937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:47.484937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:47.484937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:47.484937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:47.484937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:47.484937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:47.484937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:47.484937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:47.484937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:47.484937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:47.484937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:47.484937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:47.484937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:47.484937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:47.484937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:47.484937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:47.484937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:47.486060 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:47.486060 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:47.486060 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:47.486060 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:47.487128 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e367e8-347f-4ac0-8234-e883ba082b9f None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:47.488739 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 440/898] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:47.492466 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147deb41-d763-489e-b121-faf631ecbaf9 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:47.497616 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 459/899] 10.222.0.26 () {56 vars in 1251 bytes} [Mon May 9 09:34:47 2022] GET /identity/v3/endpoints?service_id=c73fb2bd07a1424e9df7d2892097e082&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:47.503018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e367e8-347f-4ac0-8234-e883ba082b9f None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:47.594617 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147deb41-d763-489e-b121-faf631ecbaf9 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:47.603973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e367e8-347f-4ac0-8234-e883ba082b9f None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:47.631691 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614c547-c7a4-4b57-b5ab-2c87a4e782c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:47.632358 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614c547-c7a4-4b57-b5ab-2c87a4e782c1 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:47.632643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614c547-c7a4-4b57-b5ab-2c87a4e782c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:47.632927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614c547-c7a4-4b57-b5ab-2c87a4e782c1 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:47.633755 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614c547-c7a4-4b57-b5ab-2c87a4e782c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:47.634063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614c547-c7a4-4b57-b5ab-2c87a4e782c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:47.635997 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614c547-c7a4-4b57-b5ab-2c87a4e782c1 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:47.646079 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3d4a7-61f9-4f0b-b62a-1845b6bb2856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:47.647177 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3d4a7-61f9-4f0b-b62a-1845b6bb2856 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:47.647634 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3d4a7-61f9-4f0b-b62a-1845b6bb2856 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:47.648079 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3d4a7-61f9-4f0b-b62a-1845b6bb2856 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:47.648391 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 441/900] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:47.653346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3d4a7-61f9-4f0b-b62a-1845b6bb2856 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:47.653654 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614c547-c7a4-4b57-b5ab-2c87a4e782c1 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:47.654847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3d4a7-61f9-4f0b-b62a-1845b6bb2856 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:47.669166 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 460/901] 10.222.0.26 () {56 vars in 1087 bytes} [Mon May 9 09:34:47 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:34:47.761119 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614c547-c7a4-4b57-b5ab-2c87a4e782c1 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:47.797290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff4876-54a6-489f-bcac-25232338183a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:47.798140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff4876-54a6-489f-bcac-25232338183a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:47.798415 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff4876-54a6-489f-bcac-25232338183a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:47.798682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff4876-54a6-489f-bcac-25232338183a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:47.807769 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff4876-54a6-489f-bcac-25232338183a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:47.809334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff4876-54a6-489f-bcac-25232338183a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:47.825664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-30ff4876-54a6-489f-bcac-25232338183a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 09:34:47.825664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:47.825664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:47.825664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:47.825664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:47.825664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:47.825664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:47.825664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:47.825664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:47.825664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:47.825664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:47.825664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:47.825664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:47.825664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:47.825664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:47.825664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:47.826682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:47.826682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:47.826682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:47.826682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:47.826682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:47.826682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:47.826682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:47.826682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:47.826682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:47.826682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:47.826682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:47.826682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:47.826682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:47.826682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:47.826682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:47.826682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:47.826682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:47.826682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:47.828215 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:47.828215 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:47.828215 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:47.828215 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:47.828215 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:47.828215 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:47.828215 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:47.828215 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:47.828215 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 09:34:47.828215 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:47.828215 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 442/902] 10.222.0.26 () {56 vars in 1117 bytes} [Mon May 9 09:34:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:47.832744 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3d4a7-61f9-4f0b-b62a-1845b6bb2856 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:47.938656 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3d4a7-61f9-4f0b-b62a-1845b6bb2856 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:47.971160 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875c71c-793e-4774-984f-47ff999a0290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:47.971831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875c71c-793e-4774-984f-47ff999a0290 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:47.972105 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875c71c-793e-4774-984f-47ff999a0290 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:47.972372 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875c71c-793e-4774-984f-47ff999a0290 None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:47.973238 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5875c71c-793e-4774-984f-47ff999a0290 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:47.973536 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5875c71c-793e-4774-984f-47ff999a0290 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:47.974957 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5875c71c-793e-4774-984f-47ff999a0290 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:47.993930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 461/903] 10.222.0.26 () {56 vars in 1149 bytes} [Mon May 9 09:34:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:47.999494 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff4876-54a6-489f-bcac-25232338183a None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:48.102691 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff4876-54a6-489f-bcac-25232338183a None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:48.143099 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d3976-8d8a-4316-b1ae-f486b0d8614a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:48.144020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d3976-8d8a-4316-b1ae-f486b0d8614a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:48.144298 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d3976-8d8a-4316-b1ae-f486b0d8614a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:48.144568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d3976-8d8a-4316-b1ae-f486b0d8614a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:48.212248 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d3976-8d8a-4316-b1ae-f486b0d8614a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:48.213658 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d3976-8d8a-4316-b1ae-f486b0d8614a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:48.288548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dea29-d76e-4187-bc00-4358f6cde6f2 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:48.288926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dea29-d76e-4187-bc00-4358f6cde6f2 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:48.289125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dea29-d76e-4187-bc00-4358f6cde6f2 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:48.290893 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 462/904] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:48.296225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe577a88-4937-4776-aafc-c934b275cd93 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:48.296641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe577a88-4937-4776-aafc-c934b275cd93 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:48.296985 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe577a88-4937-4776-aafc-c934b275cd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:48.314971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-fe577a88-4937-4776-aafc-c934b275cd93 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:48.316157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-fe577a88-4937-4776-aafc-c934b275cd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:48.319754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 443/905] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:34:47 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 09:34:48.324190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d3976-8d8a-4316-b1ae-f486b0d8614a None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:48.404786 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 463/906] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 09:34:48.423163 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe577a88-4937-4776-aafc-c934b275cd93 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:48.448480 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d3976-8d8a-4316-b1ae-f486b0d8614a None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:48.489211 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10230a7-d743-4a57-bed4-918d7081df7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:48.489980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10230a7-d743-4a57-bed4-918d7081df7e None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:48.490222 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10230a7-d743-4a57-bed4-918d7081df7e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:48.490508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10230a7-d743-4a57-bed4-918d7081df7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:48.501351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10230a7-d743-4a57-bed4-918d7081df7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:48.502861 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10230a7-d743-4a57-bed4-918d7081df7e None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:48.520412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-a10230a7-d743-4a57-bed4-918d7081df7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:48.520412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:48.520412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:48.520412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:48.520412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:48.520412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:48.520412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:48.520412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:48.520412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:48.520412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:48.520412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:48.520412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:48.520412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:48.520412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:48.520412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:48.521462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:48.521462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:48.521462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:48.521462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:48.521462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:48.521462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:48.521462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:48.521462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:48.521462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:48.521462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:48.521462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:48.521462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:48.521462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:48.521462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:48.521462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:48.521462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:48.521462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:48.522418 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:48.522418 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:48.522418 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:48.522418 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:48.522418 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 444/907] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:48.528814 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10230a7-d743-4a57-bed4-918d7081df7e None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:48.531419 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe577a88-4937-4776-aafc-c934b275cd93 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:48.565856 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84e99e-8b37-4164-a384-38ed3384fed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:48.566525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84e99e-8b37-4164-a384-38ed3384fed1 None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:48.567103 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84e99e-8b37-4164-a384-38ed3384fed1 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:48.567392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84e99e-8b37-4164-a384-38ed3384fed1 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:48.568375 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b84e99e-8b37-4164-a384-38ed3384fed1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:48.569942 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b84e99e-8b37-4164-a384-38ed3384fed1 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:48.596695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 464/908] 10.222.0.26 () {60 vars in 1141 bytes} [Mon May 9 09:34:48 2022] POST /identity/v3/services => generated 241 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:34:48.656542 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10230a7-d743-4a57-bed4-918d7081df7e None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:48.699983 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194a4d3-84a7-440e-afa4-54860a080463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:48.700655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194a4d3-84a7-440e-afa4-54860a080463 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:48.700955 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194a4d3-84a7-440e-afa4-54860a080463 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:48.701225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194a4d3-84a7-440e-afa4-54860a080463 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:48.701952 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194a4d3-84a7-440e-afa4-54860a080463 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:48.702252 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194a4d3-84a7-440e-afa4-54860a080463 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:48.703641 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194a4d3-84a7-440e-afa4-54860a080463 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:48.716475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 445/909] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:48.722087 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84e99e-8b37-4164-a384-38ed3384fed1 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:48.825849 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84e99e-8b37-4164-a384-38ed3384fed1 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:48.861060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773486c5-a113-4794-89af-eccde478b08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:48.861859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773486c5-a113-4794-89af-eccde478b08b None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:48.862131 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773486c5-a113-4794-89af-eccde478b08b None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:48.862395 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773486c5-a113-4794-89af-eccde478b08b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:48.869124 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773486c5-a113-4794-89af-eccde478b08b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:48.870981 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773486c5-a113-4794-89af-eccde478b08b None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:48.885223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-773486c5-a113-4794-89af-eccde478b08b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:48.885223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:48.885223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:48.885223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:48.885223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:48.885223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:48.885223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:48.885223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:48.885223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:48.885223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:48.885223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:48.885223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:48.885223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:48.885223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:48.885223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:48.886179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:48.886179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:48.886179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:48.886179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:48.886179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:48.886179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:48.886179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:48.886179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:48.886179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:48.886179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:48.886179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:48.886179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:48.886179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:48.886179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:48.886179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:48.886179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:48.886179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:48.887066 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:48.887066 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:48.887066 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:48.887066 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:48.887066 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:48.887066 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:48.887066 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:48.887066 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:48.887066 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 465/910] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:48.892488 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194a4d3-84a7-440e-afa4-54860a080463 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:48.992467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194a4d3-84a7-440e-afa4-54860a080463 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:49.026027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76005f7e-ee43-45e1-b3fc-3bd39a41b18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:49.026687 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76005f7e-ee43-45e1-b3fc-3bd39a41b18b None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:49.026964 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76005f7e-ee43-45e1-b3fc-3bd39a41b18b None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:49.027233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76005f7e-ee43-45e1-b3fc-3bd39a41b18b None admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:49.028026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76005f7e-ee43-45e1-b3fc-3bd39a41b18b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:49.028329 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76005f7e-ee43-45e1-b3fc-3bd39a41b18b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:49.029806 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76005f7e-ee43-45e1-b3fc-3bd39a41b18b None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:49.043354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 446/911] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:49.049317 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773486c5-a113-4794-89af-eccde478b08b None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:49.148549 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773486c5-a113-4794-89af-eccde478b08b None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:49.181874 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7af14d-85f9-4bea-a930-912372331328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:49.182932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7af14d-85f9-4bea-a930-912372331328 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:49.183458 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7af14d-85f9-4bea-a930-912372331328 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:49.183929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7af14d-85f9-4bea-a930-912372331328 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:49.185071 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7af14d-85f9-4bea-a930-912372331328 None admin] RBAC: Adding query filter params (role.id=e8b800faebb04b2eb2cebfb7a856d7ad, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=af62b2d3f73f482b8db4173af33752c6) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:49.185612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7af14d-85f9-4bea-a930-912372331328 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:49.187306 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7af14d-85f9-4bea-a930-912372331328 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:49.196089 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 466/912] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:34:49 2022] GET /identity/v3/role_assignments?user.id=af62b2d3f73f482b8db4173af33752c6&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=e8b800faebb04b2eb2cebfb7a856d7ad => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:49.384805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc04ac-964d-4318-8edb-a2af03158cd6 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:49.385179 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc04ac-964d-4318-8edb-a2af03158cd6 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:49.385364 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc04ac-964d-4318-8edb-a2af03158cd6 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:49.386957 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 447/913] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:49.392676 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a8055-1b0a-4893-a2f3-d2468843c30a None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:49.392987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a8055-1b0a-4893-a2f3-d2468843c30a None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:49.393235 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a8055-1b0a-4893-a2f3-d2468843c30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:49.410005 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-997a8055-1b0a-4893-a2f3-d2468843c30a None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:49.411106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-997a8055-1b0a-4893-a2f3-d2468843c30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:49.483633 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 467/914] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:34:49.501862 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc04ac-964d-4318-8edb-a2af03158cd6 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:49.603600 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc04ac-964d-4318-8edb-a2af03158cd6 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:49.637439 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0da43b-2e6d-429a-a593-363cf2802780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:49.638228 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0da43b-2e6d-429a-a593-363cf2802780 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:49.638500 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0da43b-2e6d-429a-a593-363cf2802780 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:49.638766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0da43b-2e6d-429a-a593-363cf2802780 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:49.639471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0da43b-2e6d-429a-a593-363cf2802780 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:49.640922 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0da43b-2e6d-429a-a593-363cf2802780 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:49.653196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-9f0da43b-2e6d-429a-a593-363cf2802780 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 09:34:49.653196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:49.653196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:34:49.653196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:34:49.653196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:34:49.653196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:34:49.653196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:34:49.653196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:49.653196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 09:34:49.653196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:49.653196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:34:49.653196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:49.653196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:49.653196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:49.653196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:49.653196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:49.653196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:49.654644 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:49.654644 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:49.654644 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:49.654644 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:49.654644 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:49.654644 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:49.654644 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:34:49.654644 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:34:49.654644 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:34:49.654644 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:34:49.654644 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:49.654644 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:49.654644 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:49.654644 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:49.654644 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:49.654644 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:49.655517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:49.655517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:49.655517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:49.655517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:49.655517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:49.655517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:49.655517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:49.655517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:49.655517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:49.655517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:49.655517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:34:49.655517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:49.655517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 09:34:49.655517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:49.656265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 448/915] 10.222.0.26 () {56 vars in 1114 bytes} [Mon May 9 09:34:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:49.661739 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a8055-1b0a-4893-a2f3-d2468843c30a None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:49.759930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a8055-1b0a-4893-a2f3-d2468843c30a None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:49.796655 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902f1ef-b2c9-43f3-ab4b-a5bc9f5df7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:49.797345 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902f1ef-b2c9-43f3-ab4b-a5bc9f5df7d4 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:49.797618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902f1ef-b2c9-43f3-ab4b-a5bc9f5df7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:49.797886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902f1ef-b2c9-43f3-ab4b-a5bc9f5df7d4 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:49.798594 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902f1ef-b2c9-43f3-ab4b-a5bc9f5df7d4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:49.798891 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902f1ef-b2c9-43f3-ab4b-a5bc9f5df7d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:49.800263 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902f1ef-b2c9-43f3-ab4b-a5bc9f5df7d4 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:49.808779 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 468/916] 10.222.0.26 () {56 vars in 1114 bytes} [Mon May 9 09:34:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:49.811250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0da43b-2e6d-429a-a593-363cf2802780 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:49.915032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0da43b-2e6d-429a-a593-363cf2802780 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:49.946534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657133bb-0be1-4e69-9981-2ee9ab2fe2b7 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:49.946889 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657133bb-0be1-4e69-9981-2ee9ab2fe2b7 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:49.947078 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657133bb-0be1-4e69-9981-2ee9ab2fe2b7 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:49.948701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 469/917] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:49.949149 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0fe59-405f-4d1a-b3a3-f3ae9babe883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:49.950033 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0fe59-405f-4d1a-b3a3-f3ae9babe883 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:49.950504 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0fe59-405f-4d1a-b3a3-f3ae9babe883 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:49.950960 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0fe59-405f-4d1a-b3a3-f3ae9babe883 None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:49.951869 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0fe59-405f-4d1a-b3a3-f3ae9babe883 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:49.952367 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0fe59-405f-4d1a-b3a3-f3ae9babe883 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:49.953665 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd89677-1a26-4b76-9ddb-6dc91b525502 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:49.953948 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd89677-1a26-4b76-9ddb-6dc91b525502 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:49.954221 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd89677-1a26-4b76-9ddb-6dc91b525502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:49.954595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0fe59-405f-4d1a-b3a3-f3ae9babe883 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:49.960406 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 449/918] 10.222.0.26 () {56 vars in 1114 bytes} [Mon May 9 09:34:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:49.966456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0fe59-405f-4d1a-b3a3-f3ae9babe883 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:49.971209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-6dd89677-1a26-4b76-9ddb-6dc91b525502 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:49.972400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-6dd89677-1a26-4b76-9ddb-6dc91b525502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:50.040965 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 470/919] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 09:34:50.059691 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd89677-1a26-4b76-9ddb-6dc91b525502 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:50.080537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0fe59-405f-4d1a-b3a3-f3ae9babe883 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:50.123677 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217c6d6-fe2d-4b88-946d-d8fc97d05be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:50.124762 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217c6d6-fe2d-4b88-946d-d8fc97d05be1 None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:50.125249 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217c6d6-fe2d-4b88-946d-d8fc97d05be1 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:50.125719 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217c6d6-fe2d-4b88-946d-d8fc97d05be1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:50.126658 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6217c6d6-fe2d-4b88-946d-d8fc97d05be1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:50.128220 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6217c6d6-fe2d-4b88-946d-d8fc97d05be1 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:50.150176 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 450/920] 10.222.0.26 () {60 vars in 1144 bytes} [Mon May 9 09:34:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:34:50.166422 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd89677-1a26-4b76-9ddb-6dc91b525502 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:50.212887 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2002a1-8033-41eb-8720-525b9e57bf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:50.214174 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2002a1-8033-41eb-8720-525b9e57bf5c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:50.214649 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2002a1-8033-41eb-8720-525b9e57bf5c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:50.215121 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2002a1-8033-41eb-8720-525b9e57bf5c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:50.215990 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2002a1-8033-41eb-8720-525b9e57bf5c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:50.217660 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2002a1-8033-41eb-8720-525b9e57bf5c None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:50.232322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-bf2002a1-8033-41eb-8720-525b9e57bf5c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 09:34:50.232322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:50.232322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:34:50.232322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:34:50.232322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:34:50.232322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:34:50.232322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:34:50.232322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:50.232322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 09:34:50.232322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:50.232322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:34:50.232322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:50.232322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:50.232322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:50.232322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:50.232322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:50.233692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:50.233692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:50.233692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:50.233692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:50.233692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:50.233692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:50.233692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:50.233692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:34:50.233692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:34:50.233692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:34:50.233692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:34:50.233692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:50.233692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:50.233692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:50.233692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:50.233692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:50.234573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:50.234573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:50.234573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:50.234573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:50.234573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:50.234573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:50.234573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:50.234573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:50.234573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:50.234573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:50.234573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:50.234573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:34:50.234573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:50.234573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 09:34:50.234573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:50.237674 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 471/921] 10.222.0.26 () {56 vars in 1132 bytes} [Mon May 9 09:34:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:50.243818 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217c6d6-fe2d-4b88-946d-d8fc97d05be1 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:50.357274 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217c6d6-fe2d-4b88-946d-d8fc97d05be1 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:50.394775 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec4331-bc95-4e62-ae68-daa94b24f9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:50.395438 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec4331-bc95-4e62-ae68-daa94b24f9ca None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:50.395833 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec4331-bc95-4e62-ae68-daa94b24f9ca None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:50.396135 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec4331-bc95-4e62-ae68-daa94b24f9ca None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:50.396875 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec4331-bc95-4e62-ae68-daa94b24f9ca None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:50.397174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec4331-bc95-4e62-ae68-daa94b24f9ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:50.398547 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec4331-bc95-4e62-ae68-daa94b24f9ca None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:50.403708 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 451/922] 10.222.0.26 () {56 vars in 1126 bytes} [Mon May 9 09:34:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:50.409357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2002a1-8033-41eb-8720-525b9e57bf5c None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:50.507541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2002a1-8033-41eb-8720-525b9e57bf5c None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:50.545898 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8dcad-53b6-4be1-b580-4fcf6082cef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:50.546574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8dcad-53b6-4be1-b580-4fcf6082cef5 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:50.546848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8dcad-53b6-4be1-b580-4fcf6082cef5 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:50.547115 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8dcad-53b6-4be1-b580-4fcf6082cef5 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:50.547825 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8dcad-53b6-4be1-b580-4fcf6082cef5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:50.548125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8dcad-53b6-4be1-b580-4fcf6082cef5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:50.549542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8dcad-53b6-4be1-b580-4fcf6082cef5 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:50.560439 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 472/923] 10.222.0.26 () {56 vars in 1126 bytes} [Mon May 9 09:34:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:50.566130 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec4331-bc95-4e62-ae68-daa94b24f9ca None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:50.666458 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec4331-bc95-4e62-ae68-daa94b24f9ca None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:50.700457 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37ad24-9140-4eba-98af-e38d083e694f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:50.701877 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37ad24-9140-4eba-98af-e38d083e694f None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:50.702370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37ad24-9140-4eba-98af-e38d083e694f None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:50.702869 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37ad24-9140-4eba-98af-e38d083e694f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:50.703813 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37ad24-9140-4eba-98af-e38d083e694f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:50.705418 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37ad24-9140-4eba-98af-e38d083e694f None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:50.732739 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 452/924] 10.222.0.26 () {60 vars in 1144 bytes} [Mon May 9 09:34:50 2022] POST /identity/v3/endpoints => generated 347 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:34:50.945905 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d5225-6e75-4e7e-b68b-937e4745e740 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:50.947046 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d5225-6e75-4e7e-b68b-937e4745e740 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:50.947559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d5225-6e75-4e7e-b68b-937e4745e740 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:50.949408 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 473/925] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:50.955115 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe2ba9-44ff-4446-b0c4-0af8e465786b None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:50.955959 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe2ba9-44ff-4446-b0c4-0af8e465786b None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:50.956528 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe2ba9-44ff-4446-b0c4-0af8e465786b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:50.974550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-9efe2ba9-44ff-4446-b0c4-0af8e465786b None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:50.976088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-9efe2ba9-44ff-4446-b0c4-0af8e465786b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:51.049784 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 453/926] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:34:51.072787 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d5225-6e75-4e7e-b68b-937e4745e740 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:51.279138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d5225-6e75-4e7e-b68b-937e4745e740 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:51.324865 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986950c-275b-42fa-8f94-8581049614e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:51.325716 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986950c-275b-42fa-8f94-8581049614e8 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:51.325991 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986950c-275b-42fa-8f94-8581049614e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:51.326258 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986950c-275b-42fa-8f94-8581049614e8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:51.326921 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a986950c-275b-42fa-8f94-8581049614e8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:51.328352 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a986950c-275b-42fa-8f94-8581049614e8 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:51.344722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-a986950c-275b-42fa-8f94-8581049614e8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 09:34:51.344722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:51.344722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:34:51.344722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:34:51.344722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:34:51.344722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:34:51.344722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:34:51.344722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:51.344722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 09:34:51.344722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:51.344722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:34:51.344722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:51.344722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:51.344722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:51.344722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:51.344722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:51.344722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:51.345831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:51.345831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:51.345831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:51.345831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:51.345831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:51.345831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:51.345831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:34:51.345831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:34:51.345831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:34:51.345831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:34:51.345831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:51.345831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:51.345831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:51.345831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:51.345831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:51.345831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:51.346690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:51.346690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:51.346690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:51.346690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:51.346690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:51.346690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:51.346690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:51.346690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:51.346690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:51.346690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:51.346690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:34:51.346690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:51.346690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 09:34:51.346690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:51.346690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 474/927] 10.222.0.26 () {56 vars in 1126 bytes} [Mon May 9 09:34:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:51.355097 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe2ba9-44ff-4446-b0c4-0af8e465786b None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:51.515113 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5b325-0dad-4416-8fac-cc08c4d26854 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:51.515490 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5b325-0dad-4416-8fac-cc08c4d26854 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:51.515687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5b325-0dad-4416-8fac-cc08c4d26854 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:51.520872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 475/928] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:51.522466 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da311dc-3671-4b09-8e86-eafdcf5b8bc0 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:51.522747 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da311dc-3671-4b09-8e86-eafdcf5b8bc0 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:51.523034 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da311dc-3671-4b09-8e86-eafdcf5b8bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:51.543666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe2ba9-44ff-4446-b0c4-0af8e465786b None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:51.543870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-0da311dc-3671-4b09-8e86-eafdcf5b8bc0 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:51.545040 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-0da311dc-3671-4b09-8e86-eafdcf5b8bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:51.617937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ee991-303f-4a2b-bde9-268d3ca5d47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:51.621947 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ee991-303f-4a2b-bde9-268d3ca5d47c None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:51.622359 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ee991-303f-4a2b-bde9-268d3ca5d47c None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:51.622760 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ee991-303f-4a2b-bde9-268d3ca5d47c None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:51.623509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ee991-303f-4a2b-bde9-268d3ca5d47c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:51.623809 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ee991-303f-4a2b-bde9-268d3ca5d47c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:51.625238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ee991-303f-4a2b-bde9-268d3ca5d47c None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:51.636923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 476/930] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 09:34:51.639591 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da311dc-3671-4b09-8e86-eafdcf5b8bc0 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:51.640972 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 454/930] 10.222.0.26 () {56 vars in 1122 bytes} [Mon May 9 09:34:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:51.658427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ee991-303f-4a2b-bde9-268d3ca5d47c None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:51.745794 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da311dc-3671-4b09-8e86-eafdcf5b8bc0 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:51.756393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ee991-303f-4a2b-bde9-268d3ca5d47c None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:51.778472 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ee916-816c-4477-b95b-4092792e3af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:51.779223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ee916-816c-4477-b95b-4092792e3af9 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:51.779406 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ee916-816c-4477-b95b-4092792e3af9 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:51.779656 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ee916-816c-4477-b95b-4092792e3af9 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:51.780648 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ee916-816c-4477-b95b-4092792e3af9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:51.780799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ee916-816c-4477-b95b-4092792e3af9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:51.782148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ee916-816c-4477-b95b-4092792e3af9 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:51.791758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 477/931] 10.222.0.26 () {56 vars in 1122 bytes} [Mon May 9 09:34:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:51.793160 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ee916-816c-4477-b95b-4092792e3af9 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:51.799362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380564a2-852d-46ae-9773-c8f80f1c9320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:51.800190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380564a2-852d-46ae-9773-c8f80f1c9320 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:51.800462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380564a2-852d-46ae-9773-c8f80f1c9320 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:51.800857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380564a2-852d-46ae-9773-c8f80f1c9320 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:51.813367 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380564a2-852d-46ae-9773-c8f80f1c9320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:51.824085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-380564a2-852d-46ae-9773-c8f80f1c9320 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 09:34:51.824085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:51.824085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:51.824085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:51.824085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:51.824085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:51.824085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:51.824085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:51.824085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:51.824085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:51.824085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:51.824085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:51.824085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:51.824085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:51.824085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:51.824085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:51.825296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:51.825296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:51.825296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:51.825296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:51.825296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:51.825296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:51.825296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:51.825296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:51.825296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:51.825296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:51.825296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:51.825296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:51.825296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:51.825296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:51.825296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:51.825296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:51.831226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:51.831226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:51.831226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:51.831226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:51.831226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:51.831226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:51.831226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:51.831226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:51.831226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:51.831226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:51.831226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 09:34:51.831226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:51.831226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 455/932] 10.222.0.26 () {56 vars in 1120 bytes} [Mon May 9 09:34:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:51.831965 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380564a2-852d-46ae-9773-c8f80f1c9320 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:51.898475 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ee916-816c-4477-b95b-4092792e3af9 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:51.930452 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380564a2-852d-46ae-9773-c8f80f1c9320 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:51.932300 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2eed0-0208-47a9-b6c5-ef313241200f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:51.933023 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2eed0-0208-47a9-b6c5-ef313241200f None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:51.933404 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2eed0-0208-47a9-b6c5-ef313241200f None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:51.933586 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2eed0-0208-47a9-b6c5-ef313241200f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:51.934350 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2eed0-0208-47a9-b6c5-ef313241200f None admin] RBAC: Adding query filter params (interface=public, service_id=f9eac433202442d6b8707f663c860f37, region_id=RegionOne) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:51.934654 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2eed0-0208-47a9-b6c5-ef313241200f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:51.936132 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2eed0-0208-47a9-b6c5-ef313241200f None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:51.946182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 478/933] 10.222.0.26 () {56 vars in 1251 bytes} [Mon May 9 09:34:51 2022] GET /identity/v3/endpoints?service_id=f9eac433202442d6b8707f663c860f37&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:51.947933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2eed0-0208-47a9-b6c5-ef313241200f None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:51.976252 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4d815-60f9-47e3-9a05-7f1eae0777ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:51.977037 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4d815-60f9-47e3-9a05-7f1eae0777ae None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:51.977277 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4d815-60f9-47e3-9a05-7f1eae0777ae None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:51.977544 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4d815-60f9-47e3-9a05-7f1eae0777ae None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:51.978460 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4d815-60f9-47e3-9a05-7f1eae0777ae None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:51.978762 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4d815-60f9-47e3-9a05-7f1eae0777ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:51.980779 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4d815-60f9-47e3-9a05-7f1eae0777ae None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:51.996338 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 456/934] 10.222.0.26 () {56 vars in 1115 bytes} [Mon May 9 09:34:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:51.997835 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4d815-60f9-47e3-9a05-7f1eae0777ae None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:52.068767 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2eed0-0208-47a9-b6c5-ef313241200f None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:52.101964 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1cde0-1af6-4178-8e51-baa4cb6d3485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:52.102680 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1cde0-1af6-4178-8e51-baa4cb6d3485 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:52.102952 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1cde0-1af6-4178-8e51-baa4cb6d3485 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:52.103218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1cde0-1af6-4178-8e51-baa4cb6d3485 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:52.103949 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1cde0-1af6-4178-8e51-baa4cb6d3485 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:52.105356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1cde0-1af6-4178-8e51-baa4cb6d3485 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:52.110944 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4d815-60f9-47e3-9a05-7f1eae0777ae None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:52.116778 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 479/935] 10.222.0.26 () {56 vars in 1087 bytes} [Mon May 9 09:34:51 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:34:52.149215 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560f7b6-54eb-4c46-b863-f33960159b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:52.150409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560f7b6-54eb-4c46-b863-f33960159b9f None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:52.150907 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560f7b6-54eb-4c46-b863-f33960159b9f None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:52.151399 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560f7b6-54eb-4c46-b863-f33960159b9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:52.163726 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560f7b6-54eb-4c46-b863-f33960159b9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:52.165677 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560f7b6-54eb-4c46-b863-f33960159b9f None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:52.184535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-b560f7b6-54eb-4c46-b863-f33960159b9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:52.184535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:52.184535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:52.184535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:52.184535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:52.184535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:52.184535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:52.184535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:52.184535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:52.184535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:52.184535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:52.184535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:52.184535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:52.184535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:52.184535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:52.185539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:52.185539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:52.185539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:52.185539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:52.185539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:52.185539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:52.185539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:52.185539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:52.185539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:52.185539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:52.185539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:52.185539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:52.185539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:52.185539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:52.185539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:52.185539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:52.185539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:52.186427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:52.186427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:52.186427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:52.186427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:52.193021 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 457/936] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:52.195058 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1cde0-1af6-4178-8e51-baa4cb6d3485 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:52.298951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1cde0-1af6-4178-8e51-baa4cb6d3485 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:52.332330 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c6f29-64bc-4718-9891-7f1881ac75ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:52.333432 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c6f29-64bc-4718-9891-7f1881ac75ef None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:52.333906 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c6f29-64bc-4718-9891-7f1881ac75ef None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:52.334353 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c6f29-64bc-4718-9891-7f1881ac75ef None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:52.335248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c6f29-64bc-4718-9891-7f1881ac75ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:52.335722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c6f29-64bc-4718-9891-7f1881ac75ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:52.337375 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c6f29-64bc-4718-9891-7f1881ac75ef None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:52.349279 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 480/937] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:52.355302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560f7b6-54eb-4c46-b863-f33960159b9f None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:52.458008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560f7b6-54eb-4c46-b863-f33960159b9f None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:52.491611 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352fcf0-af80-4c1b-9885-5c8891385ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:52.493187 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352fcf0-af80-4c1b-9885-5c8891385ea0 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:52.493655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352fcf0-af80-4c1b-9885-5c8891385ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:52.494097 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352fcf0-af80-4c1b-9885-5c8891385ea0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:52.503375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352fcf0-af80-4c1b-9885-5c8891385ea0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:52.505124 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352fcf0-af80-4c1b-9885-5c8891385ea0 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:52.520134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-2352fcf0-af80-4c1b-9885-5c8891385ea0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 09:34:52.520134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:52.520134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:52.520134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:52.520134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:52.520134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:52.520134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:52.520134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:52.520134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:52.520134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:52.520134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:52.520134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:52.520134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:52.520134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:52.520134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:52.520134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:52.521140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:52.521140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:52.521140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:52.521140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:52.521140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:52.521140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:52.521140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:52.521140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:52.521140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:52.521140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:52.521140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:52.521140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:52.521140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:52.521140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:52.521140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:52.521140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:52.521140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:52.521140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:52.522101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:52.522101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:52.522101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:52.522101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:52.522101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:52.522101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:52.522101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:52.522101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:52.522101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 09:34:52.522101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:52.524466 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 458/938] 10.222.0.26 () {56 vars in 1117 bytes} [Mon May 9 09:34:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:52.530307 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c6f29-64bc-4718-9891-7f1881ac75ef None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:52.572703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f45c49-0cbf-4936-bb1c-747edc4543ae None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:52.573217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f45c49-0cbf-4936-bb1c-747edc4543ae None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:52.573638 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f45c49-0cbf-4936-bb1c-747edc4543ae None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:52.575377 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 459/939] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:34:52.580413 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997a26d-1f5f-4766-95a8-6d18ff3381e3 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:52.580901 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997a26d-1f5f-4766-95a8-6d18ff3381e3 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:52.581330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997a26d-1f5f-4766-95a8-6d18ff3381e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:52.599513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-e997a26d-1f5f-4766-95a8-6d18ff3381e3 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:52.600956 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e997a26d-1f5f-4766-95a8-6d18ff3381e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:52.646584 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c6f29-64bc-4718-9891-7f1881ac75ef None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:52.676818 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 460/940] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 09:34:52.686833 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721ffa0-ae47-4ee5-965e-314f7cddae5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:52.687766 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721ffa0-ae47-4ee5-965e-314f7cddae5b None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:52.688189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721ffa0-ae47-4ee5-965e-314f7cddae5b None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:52.688682 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721ffa0-ae47-4ee5-965e-314f7cddae5b None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:52.689727 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2721ffa0-ae47-4ee5-965e-314f7cddae5b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:52.690188 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2721ffa0-ae47-4ee5-965e-314f7cddae5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:52.691764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2721ffa0-ae47-4ee5-965e-314f7cddae5b None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:52.699815 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997a26d-1f5f-4766-95a8-6d18ff3381e3 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:52.710747 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 481/941] 10.222.0.26 () {56 vars in 1149 bytes} [Mon May 9 09:34:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:52.716453 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721ffa0-ae47-4ee5-965e-314f7cddae5b None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:52.806042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997a26d-1f5f-4766-95a8-6d18ff3381e3 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:52.816434 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721ffa0-ae47-4ee5-965e-314f7cddae5b None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:52.843745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b983e2b-9d77-47b8-b1e6-16f458e4456b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:52.844419 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b983e2b-9d77-47b8-b1e6-16f458e4456b None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:52.844731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b983e2b-9d77-47b8-b1e6-16f458e4456b None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:52.845007 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b983e2b-9d77-47b8-b1e6-16f458e4456b None admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:52.845772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b983e2b-9d77-47b8-b1e6-16f458e4456b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:52.847257 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b983e2b-9d77-47b8-b1e6-16f458e4456b None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:52.848758 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: WARNING keystone.server.flask.common [None req-9b983e2b-9d77-47b8-b1e6-16f458e4456b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 09 09:34:52.853409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cccbab5-c17b-4956-a110-6277bbb48ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:52.854492 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccbab5-c17b-4956-a110-6277bbb48ff0 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:52.854913 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccbab5-c17b-4956-a110-6277bbb48ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:52.855325 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccbab5-c17b-4956-a110-6277bbb48ff0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:52.867185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cccbab5-c17b-4956-a110-6277bbb48ff0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:52.869112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cccbab5-c17b-4956-a110-6277bbb48ff0 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:52.870861 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.sql.core [None req-9b983e2b-9d77-47b8-b1e6-16f458e4456b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 09:34:52.870861 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 09 09:34:52.870861 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [parameters: {'id': '46e8ee49c05e43348be5e96a42b9a2a5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 09 09:34:52.870861 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86597) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 09:34:52.888628 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-9b983e2b-9d77-47b8-b1e6-16f458e4456b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 46e8ee49c05e43348be5e96a42b9a2a5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 46e8ee49c05e43348be5e96a42b9a2a5. May 09 09:34:52.888628 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:52.888628 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 09:34:52.888628 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 09:34:52.888628 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 09:34:52.888628 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 09:34:52.888628 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 09:34:52.888628 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application result = self._query(query) May 09 09:34:52.888628 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 09:34:52.888628 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application conn.query(q) May 09 09:34:52.888628 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 09:34:52.888628 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 09:34:52.888628 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 09:34:52.889720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application result.read() May 09 09:34:52.889720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 09:34:52.889720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 09:34:52.889720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 09:34:52.889720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 09:34:52.889720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 09:34:52.889720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 09:34:52.889720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 09:34:52.889720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 09:34:52.889720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 09:34:52.889720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:52.889720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 09:34:52.889720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:52.889720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:52.889720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 09:34:52.889720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 09:34:52.889720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 09 09:34:52.889720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return project_ref.to_dict() May 09 09:34:52.890913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 09:34:52.890913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application next(self.gen) May 09 09:34:52.890913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 09:34:52.890913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application yield resource May 09 09:34:52.890913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 09:34:52.890913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application next(self.gen) May 09 09:34:52.890913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 09:34:52.890913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self.session.rollback() May 09 09:34:52.890913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 09:34:52.890913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self.force_reraise() May 09 09:34:52.890913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 09:34:52.890913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise self.value May 09 09:34:52.890913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 09:34:52.890913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 09:34:52.890913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 09:34:52.890913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application session.commit() May 09 09:34:52.890913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 09:34:52.892059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 09:34:52.892059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 09:34:52.892059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._prepare_impl() May 09 09:34:52.892059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 09:34:52.892059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self.session.flush() May 09 09:34:52.892059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 09:34:52.892059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._flush(objects) May 09 09:34:52.892059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 09:34:52.892059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 09:34:52.892059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 09:34:52.892059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application compat.raise_( May 09 09:34:52.892059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 09:34:52.892059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception May 09 09:34:52.892059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 09:34:52.892059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application flush_context.execute() May 09 09:34:52.892059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 09:34:52.892059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rec.execute(self) May 09 09:34:52.893499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 09:34:52.893499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 09:34:52.893499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 09:34:52.893499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 09:34:52.893499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 09 09:34:52.893499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application c = connection._execute_20( May 09 09:34:52.893499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 09:34:52.893499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 09:34:52.893499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 09:34:52.893499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 09:34:52.893499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 09:34:52.893499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 09:34:52.893499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 09:34:52.893499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 09:34:52.893499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 09:34:52.895106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 09:34:52.895106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 09:34:52.895106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception May 09 09:34:52.895106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 09:34:52.895106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 09:34:52.895106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 09:34:52.895106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 09:34:52.895106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 09:34:52.895106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application result = self._query(query) May 09 09:34:52.895106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 09:34:52.895106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application conn.query(q) May 09 09:34:52.895106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 09:34:52.895106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 09:34:52.895106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 09:34:52.895106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application result.read() May 09 09:34:52.895106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 09:34:52.895106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 09:34:52.896975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 09:34:52.896975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 09:34:52.896975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 09:34:52.896975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 09:34:52.896975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 09:34:52.896975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 09:34:52.896975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 09:34:52.896975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 09 09:34:52.896975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [parameters: {'id': '46e8ee49c05e43348be5e96a42b9a2a5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 09 09:34:52.896975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 09:34:52.896975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:52.896975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:34:52.896975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:52.896975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:52.896975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 09 09:34:52.898727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 09 09:34:52.898727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 09:34:52.898727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 09:34:52.898727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 09 09:34:52.898727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:52.898727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:34:52.898727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:52.898727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:52.898727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:52.898727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:52.898727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:52.898727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:52.898727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:52.898727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:52.898727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:52.898727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:52.898727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:52.906675 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:52.906675 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 09 09:34:52.906675 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 09 09:34:52.906675 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:52.906675 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:52.906675 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 09 09:34:52.906675 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.Conflict( May 09 09:34:52.906675 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 46e8ee49c05e43348be5e96a42b9a2a5. May 09 09:34:52.906675 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:52.906675 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 461/942] 10.222.0.26 () {60 vars in 1140 bytes} [Mon May 9 09:34:52 2022] POST /identity/v3/projects => generated 262 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:52.906675 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b983e2b-9d77-47b8-b1e6-16f458e4456b None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:52.907763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-9cccbab5-c17b-4956-a110-6277bbb48ff0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:52.907763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:52.907763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:52.907763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:52.907763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:52.907763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:52.907763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:52.907763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:52.907763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:52.907763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:52.907763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:52.907763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:52.907763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:52.907763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:52.907763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:52.909346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:52.909346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:52.909346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:52.909346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:52.909346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:52.909346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:52.909346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:52.909346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:52.909346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:52.909346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:52.909346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:52.909346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:52.909346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:52.909346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:52.909346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:52.909346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:52.909346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:52.913000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:52.913000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:52.913000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:52.913000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:52.913000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 482/943] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:52.913000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cccbab5-c17b-4956-a110-6277bbb48ff0 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:53.001288 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b983e2b-9d77-47b8-b1e6-16f458e4456b None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:53.010208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cccbab5-c17b-4956-a110-6277bbb48ff0 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:53.034379 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835c5a2-671e-4a30-9f0e-02dac8925401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:53.035173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835c5a2-671e-4a30-9f0e-02dac8925401 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:53.035447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835c5a2-671e-4a30-9f0e-02dac8925401 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:53.035714 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835c5a2-671e-4a30-9f0e-02dac8925401 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:53.042594 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835c5a2-671e-4a30-9f0e-02dac8925401 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:53.044238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835c5a2-671e-4a30-9f0e-02dac8925401 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:53.045115 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58561f-5cbd-4221-a024-ae144fa7e043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:53.045796 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58561f-5cbd-4221-a024-ae144fa7e043 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:53.046055 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58561f-5cbd-4221-a024-ae144fa7e043 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:53.046323 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58561f-5cbd-4221-a024-ae144fa7e043 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:53.047044 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58561f-5cbd-4221-a024-ae144fa7e043 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:53.047342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58561f-5cbd-4221-a024-ae144fa7e043 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:53.048713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58561f-5cbd-4221-a024-ae144fa7e043 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:53.058275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-3835c5a2-671e-4a30-9f0e-02dac8925401 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:53.058275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:53.058275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:53.058275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:53.058275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:53.058275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:53.058275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:53.058275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:53.058275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:53.058275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:53.058275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:53.058275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:53.058275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:53.058275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:53.058275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:53.059203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:53.059203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:53.059203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:53.059203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:53.059203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:53.059203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:53.059203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:53.059203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:53.059203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:53.059203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:53.059203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:53.059203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:53.059203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:53.059203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:53.059203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:53.059203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:53.059203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:53.060119 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:53.060119 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:53.060119 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:53.060119 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:53.060119 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:53.060119 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:53.060119 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:53.060119 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:53.064306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 462/944] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:53.064428 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 483/945] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:53.065069 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58561f-5cbd-4221-a024-ae144fa7e043 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:53.065585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd83df-cf2d-44ba-a91a-6b7e4d563a13 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:53.065867 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd83df-cf2d-44ba-a91a-6b7e4d563a13 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:53.066137 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd83df-cf2d-44ba-a91a-6b7e4d563a13 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:53.072763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 463/946] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:53.072763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd83df-cf2d-44ba-a91a-6b7e4d563a13 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:53.162164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58561f-5cbd-4221-a024-ae144fa7e043 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:53.168188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd83df-cf2d-44ba-a91a-6b7e4d563a13 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:53.194377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215f43c-abf6-4a6b-8e59-26a0fba44a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:53.195160 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215f43c-abf6-4a6b-8e59-26a0fba44a13 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:53.195460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215f43c-abf6-4a6b-8e59-26a0fba44a13 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:53.195740 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215f43c-abf6-4a6b-8e59-26a0fba44a13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:53.202642 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215f43c-abf6-4a6b-8e59-26a0fba44a13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:53.203223 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f792368-5ae7-4ae2-aba2-069c55bc3cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:53.203882 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f792368-5ae7-4ae2-aba2-069c55bc3cdc None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:53.204077 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215f43c-abf6-4a6b-8e59-26a0fba44a13 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:53.204194 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f792368-5ae7-4ae2-aba2-069c55bc3cdc None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:53.204453 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f792368-5ae7-4ae2-aba2-069c55bc3cdc None admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:53.205295 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f792368-5ae7-4ae2-aba2-069c55bc3cdc None admin] RBAC: Adding query filter params (name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:53.205604 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f792368-5ae7-4ae2-aba2-069c55bc3cdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:53.207019 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f792368-5ae7-4ae2-aba2-069c55bc3cdc None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:53.218574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-9215f43c-abf6-4a6b-8e59-26a0fba44a13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:53.218574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:53.218574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:53.218574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:53.218574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:53.218574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:53.218574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:53.218574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:53.218574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:53.218574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:53.218574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:53.218574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:53.218574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:53.218574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:53.218574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:53.219483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:53.219483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:53.219483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:53.219483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:53.219483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:53.219483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:53.219483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:53.219483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:53.219483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:53.219483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:53.219483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:53.219483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:53.219483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:53.219483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:53.219483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:53.219483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:53.219483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:53.225831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:53.225831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:53.225831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:53.225831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:53.225831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:53.225831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:53.225831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:53.225831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:53.225831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 484/947] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:53.225831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c895c58-d281-4e12-96ec-eac6d0a00764 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:53.225831 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c895c58-d281-4e12-96ec-eac6d0a00764 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:53.226443 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c895c58-d281-4e12-96ec-eac6d0a00764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:53.229347 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 464/948] 10.222.0.26 () {56 vars in 1112 bytes} [Mon May 9 09:34:53 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:53.237256 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f792368-5ae7-4ae2-aba2-069c55bc3cdc None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:53.244733 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-9c895c58-d281-4e12-96ec-eac6d0a00764 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:53.246890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-9c895c58-d281-4e12-96ec-eac6d0a00764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:53.337121 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 485/949] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:34:53.352629 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f792368-5ae7-4ae2-aba2-069c55bc3cdc None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:53.353745 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c895c58-d281-4e12-96ec-eac6d0a00764 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:53.391150 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20429c93-7514-4710-9e40-e20fb5633c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:53.391824 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20429c93-7514-4710-9e40-e20fb5633c22 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:53.392100 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20429c93-7514-4710-9e40-e20fb5633c22 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:53.392372 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20429c93-7514-4710-9e40-e20fb5633c22 None admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:53.393254 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20429c93-7514-4710-9e40-e20fb5633c22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:53.393559 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20429c93-7514-4710-9e40-e20fb5633c22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:53.395028 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20429c93-7514-4710-9e40-e20fb5633c22 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:53.408436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 465/950] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:53.414581 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20429c93-7514-4710-9e40-e20fb5633c22 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:53.480889 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c895c58-d281-4e12-96ec-eac6d0a00764 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:53.526843 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13433605-50a0-40be-83d4-8d99f4f07305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:53.527643 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13433605-50a0-40be-83d4-8d99f4f07305 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:53.527917 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13433605-50a0-40be-83d4-8d99f4f07305 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:53.528180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13433605-50a0-40be-83d4-8d99f4f07305 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:53.528950 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13433605-50a0-40be-83d4-8d99f4f07305 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:53.530237 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13433605-50a0-40be-83d4-8d99f4f07305 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:53.533953 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20429c93-7514-4710-9e40-e20fb5633c22 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:53.542027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-13433605-50a0-40be-83d4-8d99f4f07305 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 09:34:53.542027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:53.542027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:34:53.542027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:34:53.542027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:34:53.542027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:34:53.542027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:34:53.542027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:53.542027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 09:34:53.542027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:53.542027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:34:53.542027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:53.542027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:53.542027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:53.542027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:53.542027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:53.542027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:53.543067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:53.543067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:53.543067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:53.543067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:53.543067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:53.543067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:53.543067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:34:53.543067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:34:53.543067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:34:53.543067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:34:53.543067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:53.543067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:53.543067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:53.543067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:53.543067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:53.543067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:53.543919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:53.543919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:53.543919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:53.543919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:53.543919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:53.543919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:53.543919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:53.543919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:53.543919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:53.543919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:53.543919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:34:53.543919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:53.543919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 09:34:53.543919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:53.543919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 486/951] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:53.550280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13433605-50a0-40be-83d4-8d99f4f07305 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:53.583510 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362c01d-bef1-4528-b4ea-85bf616589ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:53.584820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362c01d-bef1-4528-b4ea-85bf616589ec None admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:53.585551 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362c01d-bef1-4528-b4ea-85bf616589ec None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:53.586141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362c01d-bef1-4528-b4ea-85bf616589ec None admin] PATH_INFO: `/v3/projects/9cd39464055141b8b43fde20d0885c99/users/af62b2d3f73f482b8db4173af33752c6/roles/e8b800faebb04b2eb2cebfb7a856d7ad` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:53.589589 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362c01d-bef1-4528-b4ea-85bf616589ec None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd39464055141b8b43fde20d0885c99, user_id=af62b2d3f73f482b8db4173af33752c6, role_id=e8b800faebb04b2eb2cebfb7a856d7ad)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:53.591339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362c01d-bef1-4528-b4ea-85bf616589ec None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:53.610371 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 466/952] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:34:53 2022] PUT /identity/v3/projects/9cd39464055141b8b43fde20d0885c99/users/af62b2d3f73f482b8db4173af33752c6/roles/e8b800faebb04b2eb2cebfb7a856d7ad => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:34:53.664917 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13433605-50a0-40be-83d4-8d99f4f07305 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:53.704248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd664c19-09b5-4821-9b03-37d3375aa8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:53.705214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd664c19-09b5-4821-9b03-37d3375aa8a0 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:53.705762 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd664c19-09b5-4821-9b03-37d3375aa8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:53.706216 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd664c19-09b5-4821-9b03-37d3375aa8a0 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:53.707079 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd664c19-09b5-4821-9b03-37d3375aa8a0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:53.707535 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd664c19-09b5-4821-9b03-37d3375aa8a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:53.709099 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd664c19-09b5-4821-9b03-37d3375aa8a0 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:53.715366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 487/953] 10.222.0.26 () {56 vars in 1112 bytes} [Mon May 9 09:34:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:53.725867 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362c01d-bef1-4528-b4ea-85bf616589ec None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:53.832727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362c01d-bef1-4528-b4ea-85bf616589ec None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:53.867550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff278672-1058-4849-9312-e42b5be0ab0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:53.868222 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff278672-1058-4849-9312-e42b5be0ab0c None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:53.868497 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff278672-1058-4849-9312-e42b5be0ab0c None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:53.868800 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff278672-1058-4849-9312-e42b5be0ab0c None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:53.869510 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff278672-1058-4849-9312-e42b5be0ab0c None admin] RBAC: Adding query filter params (type=compute) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:53.869810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff278672-1058-4849-9312-e42b5be0ab0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:53.871490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff278672-1058-4849-9312-e42b5be0ab0c None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:53.876356 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 467/954] 10.222.0.26 () {56 vars in 1112 bytes} [Mon May 9 09:34:53 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:54.405060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d340c7-1f41-4fe3-b870-32965300ef3c None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:54.415760 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d340c7-1f41-4fe3-b870-32965300ef3c None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:54.415760 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d340c7-1f41-4fe3-b870-32965300ef3c None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:54.415760 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 488/955] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:54.416026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71b822-b2e2-4a61-bb25-d2fcb4b60cd6 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:54.416026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71b822-b2e2-4a61-bb25-d2fcb4b60cd6 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:54.416026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71b822-b2e2-4a61-bb25-d2fcb4b60cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:54.432168 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-6d71b822-b2e2-4a61-bb25-d2fcb4b60cd6 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:54.434053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-6d71b822-b2e2-4a61-bb25-d2fcb4b60cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:54.507002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 468/956] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:34:54.529685 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d340c7-1f41-4fe3-b870-32965300ef3c None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:54.641597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d340c7-1f41-4fe3-b870-32965300ef3c None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:54.674672 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a358ee5-def2-4c42-a0de-8c1f55e85086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:54.675815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a358ee5-def2-4c42-a0de-8c1f55e85086 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:54.676297 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a358ee5-def2-4c42-a0de-8c1f55e85086 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:54.676790 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a358ee5-def2-4c42-a0de-8c1f55e85086 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:54.677666 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a358ee5-def2-4c42-a0de-8c1f55e85086 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:54.679245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a358ee5-def2-4c42-a0de-8c1f55e85086 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:54.691460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-7a358ee5-def2-4c42-a0de-8c1f55e85086 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 09:34:54.691460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:54.691460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:34:54.691460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:34:54.691460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:34:54.691460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:34:54.691460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:34:54.691460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:54.691460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 09:34:54.691460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:54.691460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:34:54.691460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:54.691460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:54.691460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:54.691460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:54.691460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:54.691460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:54.692687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:54.692687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:54.692687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:54.692687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:54.692687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:54.692687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:54.692687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:34:54.692687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:34:54.692687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:34:54.692687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:34:54.692687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:54.692687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:54.692687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:54.692687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:54.692687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:54.692687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:54.693622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:54.693622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:54.693622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:54.693622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:54.693622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:54.693622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:54.693622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:54.693622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:54.693622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:54.693622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:54.693622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:34:54.693622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:54.693622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 09:34:54.693622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:54.696648 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 489/957] 10.222.0.26 () {56 vars in 1126 bytes} [Mon May 9 09:34:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:54.702837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d71b822-b2e2-4a61-bb25-d2fcb4b60cd6 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:54.799123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d71b822-b2e2-4a61-bb25-d2fcb4b60cd6 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:54.837082 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c5fa5-f291-418d-bcf8-3f4f9a677661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:54.837741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c5fa5-f291-418d-bcf8-3f4f9a677661 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:54.838013 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c5fa5-f291-418d-bcf8-3f4f9a677661 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:54.838276 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c5fa5-f291-418d-bcf8-3f4f9a677661 None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:54.838970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c5fa5-f291-418d-bcf8-3f4f9a677661 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:54.839266 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c5fa5-f291-418d-bcf8-3f4f9a677661 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:54.840627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c5fa5-f291-418d-bcf8-3f4f9a677661 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:54.845664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 469/958] 10.222.0.26 () {56 vars in 1122 bytes} [Mon May 9 09:34:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:54.851179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a358ee5-def2-4c42-a0de-8c1f55e85086 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:54.946298 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a358ee5-def2-4c42-a0de-8c1f55e85086 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:54.978802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1bd825-4830-4a8b-b0f6-5381f73c77fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:54.979851 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1bd825-4830-4a8b-b0f6-5381f73c77fe None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:54.980346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1bd825-4830-4a8b-b0f6-5381f73c77fe None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:54.980858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1bd825-4830-4a8b-b0f6-5381f73c77fe None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:54.981795 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1bd825-4830-4a8b-b0f6-5381f73c77fe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:54.982316 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1bd825-4830-4a8b-b0f6-5381f73c77fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:54.983951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1bd825-4830-4a8b-b0f6-5381f73c77fe None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:54.990284 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 490/959] 10.222.0.26 () {56 vars in 1122 bytes} [Mon May 9 09:34:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:54.996325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c5fa5-f291-418d-bcf8-3f4f9a677661 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:55.092798 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c5fa5-f291-418d-bcf8-3f4f9a677661 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:55.126377 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b37fc3-746f-45a9-b433-16236cf63d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:55.127035 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b37fc3-746f-45a9-b433-16236cf63d41 None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:55.127308 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b37fc3-746f-45a9-b433-16236cf63d41 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:55.127572 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b37fc3-746f-45a9-b433-16236cf63d41 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:55.128318 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b37fc3-746f-45a9-b433-16236cf63d41 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:55.129708 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b37fc3-746f-45a9-b433-16236cf63d41 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:55.151025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 470/960] 10.222.0.26 () {60 vars in 1144 bytes} [Mon May 9 09:34:54 2022] POST /identity/v3/endpoints => generated 349 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:34:55.628085 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870c6e3-9b6b-4f18-b7bf-da313c59825b None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:55.628674 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870c6e3-9b6b-4f18-b7bf-da313c59825b None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:55.628907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870c6e3-9b6b-4f18-b7bf-da313c59825b None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:55.630616 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 491/961] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:55.636134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c971d1-beb7-49d8-989a-d928f57ad91c None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:55.636621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c971d1-beb7-49d8-989a-d928f57ad91c None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:55.636910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c971d1-beb7-49d8-989a-d928f57ad91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:55.654095 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-e7c971d1-beb7-49d8-989a-d928f57ad91c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:55.655275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e7c971d1-beb7-49d8-989a-d928f57ad91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:55.728069 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 471/962] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:34:55.752497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870c6e3-9b6b-4f18-b7bf-da313c59825b None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:55.858756 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870c6e3-9b6b-4f18-b7bf-da313c59825b None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:55.892361 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d0ceb-c04e-4cba-906c-e7dee7611bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:55.893432 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d0ceb-c04e-4cba-906c-e7dee7611bea None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:55.893928 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d0ceb-c04e-4cba-906c-e7dee7611bea None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:55.894414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d0ceb-c04e-4cba-906c-e7dee7611bea None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:55.895332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d0ceb-c04e-4cba-906c-e7dee7611bea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:55.897035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d0ceb-c04e-4cba-906c-e7dee7611bea None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:55.898690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: WARNING keystone.server.flask.common [None req-4e8d0ceb-c04e-4cba-906c-e7dee7611bea None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 09 09:34:55.916265 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.sql.core [None req-4e8d0ceb-c04e-4cba-906c-e7dee7611bea None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 09:34:55.916265 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 09 09:34:55.916265 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [parameters: {'user_id': '7c8c00622c5c4705859cb34597909c4e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 09 09:34:55.916265 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86598) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 09:34:55.925206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-4e8d0ceb-c04e-4cba-906c-e7dee7611bea None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 09 09:34:55.925206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:55.925206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 09:34:55.925206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 09:34:55.925206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 09:34:55.925206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 09:34:55.925206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 09:34:55.925206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application result = self._query(query) May 09 09:34:55.925206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 09:34:55.925206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application conn.query(q) May 09 09:34:55.925206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 09:34:55.925206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 09:34:55.925206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 09:34:55.925206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application result.read() May 09 09:34:55.925206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 09:34:55.926413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 09:34:55.926413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 09:34:55.926413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 09:34:55.926413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 09:34:55.926413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 09:34:55.926413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 09:34:55.926413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 09:34:55.926413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 09:34:55.926413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:55.926413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 09:34:55.926413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:55.926413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:55.926413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 09:34:55.926413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 09:34:55.926413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 09 09:34:55.926413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 09 09:34:55.926413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 09:34:55.926413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application next(self.gen) May 09 09:34:55.927657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 09:34:55.927657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application yield resource May 09 09:34:55.927657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 09:34:55.927657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application next(self.gen) May 09 09:34:55.927657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 09:34:55.927657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self.session.rollback() May 09 09:34:55.927657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 09:34:55.927657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self.force_reraise() May 09 09:34:55.927657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 09:34:55.927657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise self.value May 09 09:34:55.927657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 09:34:55.927657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 09:34:55.927657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 09:34:55.927657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application session.commit() May 09 09:34:55.927657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 09:34:55.927657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 09:34:55.927657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 09:34:55.928802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._prepare_impl() May 09 09:34:55.928802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 09:34:55.928802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self.session.flush() May 09 09:34:55.928802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 09:34:55.928802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._flush(objects) May 09 09:34:55.928802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 09:34:55.928802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 09:34:55.928802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 09:34:55.928802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application compat.raise_( May 09 09:34:55.928802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 09:34:55.928802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception May 09 09:34:55.928802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 09:34:55.928802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application flush_context.execute() May 09 09:34:55.928802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 09:34:55.928802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rec.execute(self) May 09 09:34:55.928802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 09:34:55.928802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 09:34:55.929815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 09:34:55.929815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 09:34:55.929815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 09 09:34:55.929815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application result = connection._execute_20( May 09 09:34:55.929815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 09:34:55.929815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 09:34:55.929815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 09:34:55.929815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 09:34:55.929815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 09:34:55.929815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 09:34:55.929815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 09:34:55.929815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 09:34:55.929815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 09:34:55.929815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 09:34:55.929815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 09:34:55.929815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception May 09 09:34:55.930955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 09:34:55.930955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 09:34:55.930955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 09:34:55.930955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 09:34:55.930955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 09:34:55.930955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application result = self._query(query) May 09 09:34:55.930955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 09:34:55.930955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application conn.query(q) May 09 09:34:55.930955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 09:34:55.930955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 09:34:55.930955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 09:34:55.930955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application result.read() May 09 09:34:55.930955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 09:34:55.930955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 09:34:55.930955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 09:34:55.930955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 09:34:55.932056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 09:34:55.932056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 09:34:55.932056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 09:34:55.932056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 09:34:55.932056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 09:34:55.932056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 09 09:34:55.932056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [parameters: {'user_id': '7c8c00622c5c4705859cb34597909c4e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 09 09:34:55.932056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 09:34:55.932056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:55.932056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:34:55.932056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:55.932056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:55.932056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:55.932056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:55.932056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:55.933049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:55.933049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:55.933049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:55.933049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:55.933049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:55.933049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:55.933049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:55.933049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 09 09:34:55.933049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 09 09:34:55.933049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:55.933049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:55.933049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:55.933049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:55.933049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:55.933049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:55.933049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 09 09:34:55.933049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 09 09:34:55.934000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 09 09:34:55.934000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 09 09:34:55.934000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 09:34:55.934000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 09:34:55.934000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 09 09:34:55.934000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:55.936418 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 492/963] 10.222.0.26 () {60 vars in 1131 bytes} [Mon May 9 09:34:55 2022] POST /identity/v3/users => generated 160 bytes in 184 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:55.942044 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c971d1-beb7-49d8-989a-d928f57ad91c None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:55.955638 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6e987-c044-4d0c-b05a-e6434ad5a74a None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:55.956222 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6e987-c044-4d0c-b05a-e6434ad5a74a None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:55.956801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6e987-c044-4d0c-b05a-e6434ad5a74a None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:55.961722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 493/964] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:34:55.967096 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77112ae3-e47c-48c7-8d58-360134815a56 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:55.967667 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77112ae3-e47c-48c7-8d58-360134815a56 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:55.968127 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77112ae3-e47c-48c7-8d58-360134815a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:55.985615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-77112ae3-e47c-48c7-8d58-360134815a56 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:55.987419 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-77112ae3-e47c-48c7-8d58-360134815a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:56.039925 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c971d1-beb7-49d8-989a-d928f57ad91c None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:56.060983 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 494/965] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 09:34:56.073083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51823f4c-f423-4ecf-a8c8-5d6ed3809859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:56.073999 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51823f4c-f423-4ecf-a8c8-5d6ed3809859 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:56.074351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51823f4c-f423-4ecf-a8c8-5d6ed3809859 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:56.074680 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51823f4c-f423-4ecf-a8c8-5d6ed3809859 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:56.079521 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77112ae3-e47c-48c7-8d58-360134815a56 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:56.082089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51823f4c-f423-4ecf-a8c8-5d6ed3809859 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:56.083608 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51823f4c-f423-4ecf-a8c8-5d6ed3809859 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:56.098170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-51823f4c-f423-4ecf-a8c8-5d6ed3809859 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 09:34:56.098170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:56.098170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:56.098170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:56.098170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:56.098170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:56.098170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:56.098170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:56.098170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:56.098170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:56.098170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:56.098170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:56.098170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:56.098170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:56.098170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:56.098170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:56.099186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:56.099186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:56.099186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:56.099186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:56.099186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:56.099186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:56.099186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:56.099186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:56.099186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:56.099186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:56.099186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:56.099186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:56.099186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:56.099186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:56.099186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:56.099186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:56.099186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:56.099186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:56.100185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:56.100185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:56.100185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:56.100185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:56.100185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:56.100185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:56.100185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:56.100185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:56.100185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 09:34:56.100185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:56.100185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 472/966] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:34:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:56.103834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233b79e-c201-42f4-bc8a-2d5ca3fe39c8 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:56.104030 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233b79e-c201-42f4-bc8a-2d5ca3fe39c8 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:56.104268 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233b79e-c201-42f4-bc8a-2d5ca3fe39c8 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:56.105785 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 473/967] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:56.106439 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233b79e-c201-42f4-bc8a-2d5ca3fe39c8 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:56.182618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77112ae3-e47c-48c7-8d58-360134815a56 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:56.211552 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233b79e-c201-42f4-bc8a-2d5ca3fe39c8 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:56.215716 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6824753-47a7-4655-b305-9fab644f43f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:56.216529 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6824753-47a7-4655-b305-9fab644f43f9 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:56.216837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6824753-47a7-4655-b305-9fab644f43f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:56.217248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6824753-47a7-4655-b305-9fab644f43f9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:56.227987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6824753-47a7-4655-b305-9fab644f43f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:34:56.236134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-c6824753-47a7-4655-b305-9fab644f43f9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 09:34:56.236134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:56.236134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:56.236134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:56.236134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:56.236134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:56.236134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:56.236134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:56.236134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:56.236134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:56.236134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:56.236134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:56.236134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:34:56.236134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:34:56.236134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:34:56.236134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:34:56.237145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:34:56.237145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:34:56.237145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:34:56.237145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:34:56.237145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:56.237145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:56.237145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:56.237145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:56.237145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:56.237145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:56.237145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:56.237145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:56.237145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:56.237145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:56.237145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:56.237145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:56.238038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:56.238038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:56.238038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:56.238038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:56.238038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:34:56.238038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:34:56.238038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:34:56.238038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:34:56.238038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:34:56.238038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:34:56.238038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 09:34:56.238038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:56.238038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 495/968] 10.222.0.26 () {56 vars in 1120 bytes} [Mon May 9 09:34:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:56.238780 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1dd6fb-4797-42fa-80b8-5e0f3fab76d6 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:56.238780 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1dd6fb-4797-42fa-80b8-5e0f3fab76d6 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:56.239412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1dd6fb-4797-42fa-80b8-5e0f3fab76d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:56.249874 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef760014-4915-43ff-8bf0-5e76d065dbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:56.250543 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef760014-4915-43ff-8bf0-5e76d065dbd7 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:56.250820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef760014-4915-43ff-8bf0-5e76d065dbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:56.251078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef760014-4915-43ff-8bf0-5e76d065dbd7 None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:56.251888 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef760014-4915-43ff-8bf0-5e76d065dbd7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:56.252187 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef760014-4915-43ff-8bf0-5e76d065dbd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:56.253701 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef760014-4915-43ff-8bf0-5e76d065dbd7 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:56.256987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-6d1dd6fb-4797-42fa-80b8-5e0f3fab76d6 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:56.258417 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-6d1dd6fb-4797-42fa-80b8-5e0f3fab76d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:56.287637 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 474/969] 10.222.0.26 () {56 vars in 1101 bytes} [Mon May 9 09:34:56 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:56.288843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef760014-4915-43ff-8bf0-5e76d065dbd7 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:56.336687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 496/970] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:34:56.357463 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1dd6fb-4797-42fa-80b8-5e0f3fab76d6 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:56.406515 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef760014-4915-43ff-8bf0-5e76d065dbd7 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:56.441967 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8ed6c-548a-4ab0-bd5f-57bf83792bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:56.442555 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8ed6c-548a-4ab0-bd5f-57bf83792bcc None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:56.442831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8ed6c-548a-4ab0-bd5f-57bf83792bcc None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:56.443099 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8ed6c-548a-4ab0-bd5f-57bf83792bcc None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:56.443867 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb8ed6c-548a-4ab0-bd5f-57bf83792bcc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:56.444168 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb8ed6c-548a-4ab0-bd5f-57bf83792bcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:56.445655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb8ed6c-548a-4ab0-bd5f-57bf83792bcc None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:56.454827 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 475/971] 10.222.0.26 () {56 vars in 1115 bytes} [Mon May 9 09:34:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:56.460128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8ed6c-548a-4ab0-bd5f-57bf83792bcc None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:56.482063 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1dd6fb-4797-42fa-80b8-5e0f3fab76d6 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:56.533491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc6191-df32-45f6-bc02-aa406e5b2467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:56.533720 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc6191-df32-45f6-bc02-aa406e5b2467 None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:56.533812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc6191-df32-45f6-bc02-aa406e5b2467 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:56.534148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc6191-df32-45f6-bc02-aa406e5b2467 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:56.537269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc6191-df32-45f6-bc02-aa406e5b2467 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:56.537269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc6191-df32-45f6-bc02-aa406e5b2467 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:56.555027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 497/972] 10.222.0.26 () {60 vars in 1141 bytes} [Mon May 9 09:34:56 2022] POST /identity/v3/services => generated 242 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:34:56.591731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8ed6c-548a-4ab0-bd5f-57bf83792bcc None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:56.640972 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6004258-e3b4-4e02-8c62-9078ed022a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:56.642650 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6004258-e3b4-4e02-8c62-9078ed022a32 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:56.643129 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6004258-e3b4-4e02-8c62-9078ed022a32 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:56.643675 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6004258-e3b4-4e02-8c62-9078ed022a32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:56.657097 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6004258-e3b4-4e02-8c62-9078ed022a32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:56.658943 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6004258-e3b4-4e02-8c62-9078ed022a32 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:56.677387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-a6004258-e3b4-4e02-8c62-9078ed022a32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:56.677387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:56.677387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:56.677387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:56.677387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:56.677387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:56.677387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:56.677387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:56.677387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:56.677387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:56.677387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:56.677387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:56.677387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:56.677387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:56.677387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:56.678537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:56.678537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:56.678537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:56.678537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:56.678537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:56.678537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:56.678537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:56.678537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:56.678537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:56.678537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:56.678537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:56.678537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:56.678537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:56.678537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:56.678537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:56.678537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:56.678537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:56.679496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:56.679496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:56.679496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:56.679496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:56.686142 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 476/973] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:56.687544 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc6191-df32-45f6-bc02-aa406e5b2467 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:56.791912 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc6191-df32-45f6-bc02-aa406e5b2467 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:56.824178 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423d896-eb28-4b92-82c1-43620a3fbcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:56.824866 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423d896-eb28-4b92-82c1-43620a3fbcc8 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:56.825143 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423d896-eb28-4b92-82c1-43620a3fbcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:56.825409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423d896-eb28-4b92-82c1-43620a3fbcc8 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:56.826124 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423d896-eb28-4b92-82c1-43620a3fbcc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:56.826420 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423d896-eb28-4b92-82c1-43620a3fbcc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:56.827774 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423d896-eb28-4b92-82c1-43620a3fbcc8 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:56.840060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 498/974] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:56.845601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6004258-e3b4-4e02-8c62-9078ed022a32 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:56.945493 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6004258-e3b4-4e02-8c62-9078ed022a32 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:56.979033 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b63cd4-32c9-4d78-80fd-5392e4c37189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:56.979837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b63cd4-32c9-4d78-80fd-5392e4c37189 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:56.980108 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b63cd4-32c9-4d78-80fd-5392e4c37189 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:56.980372 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b63cd4-32c9-4d78-80fd-5392e4c37189 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:56.987890 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b63cd4-32c9-4d78-80fd-5392e4c37189 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:56.989351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b63cd4-32c9-4d78-80fd-5392e4c37189 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:57.003702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-96b63cd4-32c9-4d78-80fd-5392e4c37189 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 09:34:57.003702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:57.003702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:57.003702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:57.003702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:57.003702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:57.003702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:57.003702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:57.003702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:57.003702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:57.003702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:57.003702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:57.003702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:34:57.003702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:34:57.003702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:34:57.003702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:34:57.004787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:57.004787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:57.004787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:34:57.004787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:57.004787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:34:57.004787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:34:57.004787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:57.004787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:57.004787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:57.004787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:57.004787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:57.004787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:57.004787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:57.004787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:57.004787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:57.004787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:57.004787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:57.004787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:57.005731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:57.005731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:57.005731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:34:57.005731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:34:57.005731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:34:57.005731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:34:57.005731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:34:57.005731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:34:57.005731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 09:34:57.005731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:57.008766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 477/975] 10.222.0.26 () {56 vars in 1117 bytes} [Mon May 9 09:34:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:57.010966 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423d896-eb28-4b92-82c1-43620a3fbcc8 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:57.110476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423d896-eb28-4b92-82c1-43620a3fbcc8 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:57.143519 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38efcf1-1fd5-4781-8044-9caad4b7f2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:57.144187 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38efcf1-1fd5-4781-8044-9caad4b7f2fa None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:57.144456 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38efcf1-1fd5-4781-8044-9caad4b7f2fa None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:57.144744 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38efcf1-1fd5-4781-8044-9caad4b7f2fa None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:57.145575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38efcf1-1fd5-4781-8044-9caad4b7f2fa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:57.145871 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38efcf1-1fd5-4781-8044-9caad4b7f2fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:57.147296 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38efcf1-1fd5-4781-8044-9caad4b7f2fa None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:57.165418 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 499/976] 10.222.0.26 () {56 vars in 1149 bytes} [Mon May 9 09:34:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:57.171001 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b63cd4-32c9-4d78-80fd-5392e4c37189 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:57.274998 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b63cd4-32c9-4d78-80fd-5392e4c37189 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:57.310184 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27553394-eed4-4297-80e3-de399295c087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:57.310829 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27553394-eed4-4297-80e3-de399295c087 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:57.311101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27553394-eed4-4297-80e3-de399295c087 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:57.311365 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27553394-eed4-4297-80e3-de399295c087 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:57.376577 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27553394-eed4-4297-80e3-de399295c087 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:57.377969 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27553394-eed4-4297-80e3-de399295c087 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:57.441190 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b687e02-77cf-48ef-af57-94859011c8b3 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:57.441461 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b687e02-77cf-48ef-af57-94859011c8b3 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:57.441706 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b687e02-77cf-48ef-af57-94859011c8b3 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:57.443069 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 500/977] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:34:57.448099 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2368726-a773-4889-af0a-11fe3a1037b0 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:57.448358 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2368726-a773-4889-af0a-11fe3a1037b0 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:57.448654 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2368726-a773-4889-af0a-11fe3a1037b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:57.465668 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-d2368726-a773-4889-af0a-11fe3a1037b0 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:57.466826 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-d2368726-a773-4889-af0a-11fe3a1037b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:57.480073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 478/978] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:34:57 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 09:34:57.481556 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27553394-eed4-4297-80e3-de399295c087 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:57.533017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 501/979] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 09:34:57.555516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2368726-a773-4889-af0a-11fe3a1037b0 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:57.600161 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27553394-eed4-4297-80e3-de399295c087 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:57.632909 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3ee21-d0cd-4141-b178-c21ea7e1948a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:57.633630 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3ee21-d0cd-4141-b178-c21ea7e1948a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:57.633910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3ee21-d0cd-4141-b178-c21ea7e1948a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:57.634181 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3ee21-d0cd-4141-b178-c21ea7e1948a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:57.644043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3ee21-d0cd-4141-b178-c21ea7e1948a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:57.645526 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3ee21-d0cd-4141-b178-c21ea7e1948a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:57.652586 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2368726-a773-4889-af0a-11fe3a1037b0 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:57.662492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-bee3ee21-d0cd-4141-b178-c21ea7e1948a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:57.662492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:57.662492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:57.662492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:57.662492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:57.662492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:57.662492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:57.662492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:57.662492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:57.662492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:57.662492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:57.662492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:57.662492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:34:57.662492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:34:57.662492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:34:57.663409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:34:57.663409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:57.663409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:57.663409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:57.663409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:57.663409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:57.663409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:57.663409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:57.663409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:57.663409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:57.663409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:57.663409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:57.663409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:57.663409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:57.663409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:57.663409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:57.663409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:57.664364 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:34:57.664364 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:34:57.664364 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:34:57.664364 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:57.664364 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 479/980] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:34:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:57.669468 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3ee21-d0cd-4141-b178-c21ea7e1948a None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:57.692597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d74f08-418f-4fcf-9785-5a6f33ffa3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:57.693438 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d74f08-418f-4fcf-9785-5a6f33ffa3c3 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:57.693720 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d74f08-418f-4fcf-9785-5a6f33ffa3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:57.693993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d74f08-418f-4fcf-9785-5a6f33ffa3c3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:57.694650 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d74f08-418f-4fcf-9785-5a6f33ffa3c3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:57.696065 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d74f08-418f-4fcf-9785-5a6f33ffa3c3 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:57.707729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-29d74f08-418f-4fcf-9785-5a6f33ffa3c3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 09:34:57.707729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:57.707729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:34:57.707729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:34:57.707729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:34:57.707729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:34:57.707729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:34:57.707729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:57.707729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 09:34:57.707729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:57.707729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:34:57.707729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:57.707729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:57.707729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:57.707729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:57.707729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:57.707729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:57.708859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:57.708859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:57.708859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:57.708859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:57.708859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:57.708859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:57.708859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:34:57.708859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:34:57.708859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:34:57.708859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:34:57.708859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:57.708859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:57.708859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:57.708859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:57.708859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:57.708859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:57.710197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:57.710197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:34:57.710197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:57.710197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:57.710197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:57.710197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:57.710197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:57.710197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:57.710197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:57.710197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:57.710197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:34:57.710197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:57.710197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 09:34:57.710197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:34:57.710197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 502/981] 10.222.0.26 () {56 vars in 1126 bytes} [Mon May 9 09:34:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:57.714963 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d74f08-418f-4fcf-9785-5a6f33ffa3c3 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:57.773001 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3ee21-d0cd-4141-b178-c21ea7e1948a None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:57.808287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c68b5b-250c-4f91-ac45-599ac3d7241f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:57.808978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c68b5b-250c-4f91-ac45-599ac3d7241f None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:57.809255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c68b5b-250c-4f91-ac45-599ac3d7241f None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:57.809513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c68b5b-250c-4f91-ac45-599ac3d7241f None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:57.810253 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c68b5b-250c-4f91-ac45-599ac3d7241f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:57.810540 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c68b5b-250c-4f91-ac45-599ac3d7241f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:57.811936 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c68b5b-250c-4f91-ac45-599ac3d7241f None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:57.812608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d74f08-418f-4fcf-9785-5a6f33ffa3c3 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:57.823658 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 480/982] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:57.829055 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c68b5b-250c-4f91-ac45-599ac3d7241f None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:57.849735 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca00c4-953a-4337-bfe0-405bf27672f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:57.850397 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca00c4-953a-4337-bfe0-405bf27672f7 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:57.850678 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca00c4-953a-4337-bfe0-405bf27672f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:57.850949 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca00c4-953a-4337-bfe0-405bf27672f7 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:57.851751 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca00c4-953a-4337-bfe0-405bf27672f7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:57.852055 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca00c4-953a-4337-bfe0-405bf27672f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:57.853553 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca00c4-953a-4337-bfe0-405bf27672f7 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:57.863270 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 503/983] 10.222.0.26 () {56 vars in 1122 bytes} [Mon May 9 09:34:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:57.864095 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca00c4-953a-4337-bfe0-405bf27672f7 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:57.922785 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c68b5b-250c-4f91-ac45-599ac3d7241f None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:57.953990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ecfff-cb45-41e7-b07c-29afdd06cb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:57.954782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ecfff-cb45-41e7-b07c-29afdd06cb0a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:57.955056 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ecfff-cb45-41e7-b07c-29afdd06cb0a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:57.955325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ecfff-cb45-41e7-b07c-29afdd06cb0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:57.957354 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca00c4-953a-4337-bfe0-405bf27672f7 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:57.961822 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ecfff-cb45-41e7-b07c-29afdd06cb0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:57.963250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ecfff-cb45-41e7-b07c-29afdd06cb0a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:57.977133 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-216ecfff-cb45-41e7-b07c-29afdd06cb0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:57.977133 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:57.977133 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:57.977133 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:57.977133 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:57.977133 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:57.977133 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:57.977133 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:57.977133 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:57.977133 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:57.977133 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:57.977133 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:57.977133 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:34:57.977133 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:34:57.977133 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:34:57.978053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:34:57.978053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:57.978053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:57.978053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:57.978053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:57.978053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:57.978053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:57.978053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:57.978053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:57.978053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:57.978053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:57.978053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:57.978053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:57.978053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:57.978053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:57.978053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:57.978053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:57.978980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:34:57.978980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:34:57.978980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:34:57.978980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:34:57.978980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:34:57.978980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:34:57.978980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:34:57.978980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:57.978980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 481/984] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:34:57.984229 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ecfff-cb45-41e7-b07c-29afdd06cb0a None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:57.992948 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61026d29-d6c2-40b3-9c37-679ce4e0a73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:57.994023 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61026d29-d6c2-40b3-9c37-679ce4e0a73f None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:57.994505 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61026d29-d6c2-40b3-9c37-679ce4e0a73f None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:57.995241 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61026d29-d6c2-40b3-9c37-679ce4e0a73f None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:57.995959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61026d29-d6c2-40b3-9c37-679ce4e0a73f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:57.996270 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61026d29-d6c2-40b3-9c37-679ce4e0a73f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:57.997682 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61026d29-d6c2-40b3-9c37-679ce4e0a73f None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:58.002704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 504/985] 10.222.0.26 () {56 vars in 1122 bytes} [Mon May 9 09:34:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:58.008291 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61026d29-d6c2-40b3-9c37-679ce4e0a73f None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:58.087119 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ecfff-cb45-41e7-b07c-29afdd06cb0a None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:58.102677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61026d29-d6c2-40b3-9c37-679ce4e0a73f None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:58.124640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456013c4-8a0b-4b56-aec7-6e4196d0e1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:58.126125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456013c4-8a0b-4b56-aec7-6e4196d0e1da None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:58.126931 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456013c4-8a0b-4b56-aec7-6e4196d0e1da None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:58.127410 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456013c4-8a0b-4b56-aec7-6e4196d0e1da None admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:58.128920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456013c4-8a0b-4b56-aec7-6e4196d0e1da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:58.129251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456013c4-8a0b-4b56-aec7-6e4196d0e1da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:58.130698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456013c4-8a0b-4b56-aec7-6e4196d0e1da None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:58.135952 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23537d5f-c731-42ef-aa06-a8d87e877309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:58.136605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23537d5f-c731-42ef-aa06-a8d87e877309 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:58.136914 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23537d5f-c731-42ef-aa06-a8d87e877309 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:58.137475 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23537d5f-c731-42ef-aa06-a8d87e877309 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:58.137975 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23537d5f-c731-42ef-aa06-a8d87e877309 None admin] RBAC: Adding query filter params (interface=admin, service_id=f9eac433202442d6b8707f663c860f37, region_id=RegionOne) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:58.138285 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23537d5f-c731-42ef-aa06-a8d87e877309 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:58.139681 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23537d5f-c731-42ef-aa06-a8d87e877309 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:58.142037 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 482/986] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:34:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:58.147885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456013c4-8a0b-4b56-aec7-6e4196d0e1da None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:58.149105 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 505/987] 10.222.0.26 () {56 vars in 1249 bytes} [Mon May 9 09:34:58 2022] GET /identity/v3/endpoints?service_id=f9eac433202442d6b8707f663c860f37&interface=admin®ion_id=RegionOne => generated 193 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:34:58.154571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23537d5f-c731-42ef-aa06-a8d87e877309 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:58.248296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456013c4-8a0b-4b56-aec7-6e4196d0e1da None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:58.255424 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23537d5f-c731-42ef-aa06-a8d87e877309 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:58.282847 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5516da49-4f4d-4c40-a058-2d72cc3cafb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:58.283496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516da49-4f4d-4c40-a058-2d72cc3cafb4 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:58.283772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516da49-4f4d-4c40-a058-2d72cc3cafb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:58.284046 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516da49-4f4d-4c40-a058-2d72cc3cafb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:58.284911 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5516da49-4f4d-4c40-a058-2d72cc3cafb4 None admin] RBAC: Adding query filter params (role.id=e8b800faebb04b2eb2cebfb7a856d7ad, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=af62b2d3f73f482b8db4173af33752c6) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:58.285221 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5516da49-4f4d-4c40-a058-2d72cc3cafb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:58.286670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5516da49-4f4d-4c40-a058-2d72cc3cafb4 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:58.291939 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6827a24-ea84-4d1d-a442-88ccf52c4765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:58.292696 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 483/988] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:34:58 2022] GET /identity/v3/role_assignments?user.id=af62b2d3f73f482b8db4173af33752c6&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=e8b800faebb04b2eb2cebfb7a856d7ad => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:58.293013 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6827a24-ea84-4d1d-a442-88ccf52c4765 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:58.293290 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6827a24-ea84-4d1d-a442-88ccf52c4765 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:58.293555 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6827a24-ea84-4d1d-a442-88ccf52c4765 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:58.294302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6827a24-ea84-4d1d-a442-88ccf52c4765 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:58.295650 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6827a24-ea84-4d1d-a442-88ccf52c4765 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:58.320720 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 506/989] 10.222.0.26 () {56 vars in 1087 bytes} [Mon May 9 09:34:58 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:34:58.790607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458e4ae-a721-4ba2-b93e-0bf1f361bf91 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:58.791197 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458e4ae-a721-4ba2-b93e-0bf1f361bf91 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:58.791197 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458e4ae-a721-4ba2-b93e-0bf1f361bf91 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:58.796729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 484/990] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:34:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:34:58.798883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09b90e-ed12-49ad-9ed1-e4837f6a0ce8 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:58.799331 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09b90e-ed12-49ad-9ed1-e4837f6a0ce8 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:58.799424 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09b90e-ed12-49ad-9ed1-e4837f6a0ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:58.818252 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-fe09b90e-ed12-49ad-9ed1-e4837f6a0ce8 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:34:58.819360 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-fe09b90e-ed12-49ad-9ed1-e4837f6a0ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:34:58.888375 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 507/991] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:34:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:34:58.910449 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458e4ae-a721-4ba2-b93e-0bf1f361bf91 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:59.011118 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458e4ae-a721-4ba2-b93e-0bf1f361bf91 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:59.043696 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a60cd-c577-450d-8c7c-1d4d84b0ece1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:59.044479 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a60cd-c577-450d-8c7c-1d4d84b0ece1 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:59.044776 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a60cd-c577-450d-8c7c-1d4d84b0ece1 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:59.045041 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a60cd-c577-450d-8c7c-1d4d84b0ece1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:59.045683 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a60cd-c577-450d-8c7c-1d4d84b0ece1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:59.047092 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a60cd-c577-450d-8c7c-1d4d84b0ece1 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:59.058593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-d15a60cd-c577-450d-8c7c-1d4d84b0ece1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 09:34:59.058593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:59.058593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:34:59.058593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:34:59.058593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:34:59.058593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:34:59.058593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:34:59.058593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:59.058593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 09:34:59.058593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:59.058593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:34:59.058593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:59.058593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:34:59.058593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:34:59.058593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:34:59.058593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:34:59.058593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:34:59.059640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:34:59.059640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:34:59.059640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:34:59.059640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:34:59.059640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:34:59.059640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:34:59.059640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:34:59.059640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:34:59.059640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:34:59.059640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:34:59.059640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:34:59.059640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:34:59.059640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:34:59.059640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:34:59.059640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:34:59.059640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:34:59.060495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:34:59.060495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:34:59.060495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:34:59.060495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:34:59.060495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:34:59.060495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:34:59.060495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:34:59.060495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:34:59.060495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:34:59.060495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:34:59.060495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:34:59.060495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:34:59.060495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 09:34:59.060495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:34:59.064769 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 485/992] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:34:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:34:59.066185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09b90e-ed12-49ad-9ed1-e4837f6a0ce8 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:59.165357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09b90e-ed12-49ad-9ed1-e4837f6a0ce8 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:59.199205 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d3252-59b7-4d31-8ba1-2c45d84509bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:59.199871 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d3252-59b7-4d31-8ba1-2c45d84509bd None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:59.200145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d3252-59b7-4d31-8ba1-2c45d84509bd None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:59.200413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d3252-59b7-4d31-8ba1-2c45d84509bd None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:59.201139 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d3252-59b7-4d31-8ba1-2c45d84509bd None admin] RBAC: Adding query filter params (name=compute) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:59.201440 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d3252-59b7-4d31-8ba1-2c45d84509bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:59.202813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d3252-59b7-4d31-8ba1-2c45d84509bd None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:59.212380 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 508/993] 10.222.0.26 () {56 vars in 1112 bytes} [Mon May 9 09:34:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:59.213596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a60cd-c577-450d-8c7c-1d4d84b0ece1 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:59.311969 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a60cd-c577-450d-8c7c-1d4d84b0ece1 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:59.344387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf999c97-0c29-40b5-aa6f-48d34fd2551d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:59.345063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf999c97-0c29-40b5-aa6f-48d34fd2551d None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:59.345337 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf999c97-0c29-40b5-aa6f-48d34fd2551d None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:59.345601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf999c97-0c29-40b5-aa6f-48d34fd2551d None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:59.346293 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf999c97-0c29-40b5-aa6f-48d34fd2551d None admin] RBAC: Adding query filter params (type=compute) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:59.346590 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf999c97-0c29-40b5-aa6f-48d34fd2551d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:59.347948 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf999c97-0c29-40b5-aa6f-48d34fd2551d None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:59.352803 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 486/994] 10.222.0.26 () {56 vars in 1112 bytes} [Mon May 9 09:34:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:59.358547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d3252-59b7-4d31-8ba1-2c45d84509bd None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:59.456162 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d3252-59b7-4d31-8ba1-2c45d84509bd None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:59.488670 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a38df-6547-4245-ba8c-e3ef7d5b00d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:59.489344 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a38df-6547-4245-ba8c-e3ef7d5b00d6 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:59.489618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a38df-6547-4245-ba8c-e3ef7d5b00d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:59.489860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a38df-6547-4245-ba8c-e3ef7d5b00d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:59.490654 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a38df-6547-4245-ba8c-e3ef7d5b00d6 None admin] RBAC: Adding query filter params (interface=public, service_id=73d41d2bbc5e4ad5b3f9de2c52ca1602, region_id=RegionOne) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:34:59.490955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a38df-6547-4245-ba8c-e3ef7d5b00d6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:59.492344 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a38df-6547-4245-ba8c-e3ef7d5b00d6 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:59.497825 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 509/995] 10.222.0.26 () {56 vars in 1251 bytes} [Mon May 9 09:34:59 2022] GET /identity/v3/endpoints?service_id=73d41d2bbc5e4ad5b3f9de2c52ca1602&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:34:59.503289 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf999c97-0c29-40b5-aa6f-48d34fd2551d None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:34:59.608233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf999c97-0c29-40b5-aa6f-48d34fd2551d None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:34:59.640499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b8ad7-bf08-48ab-a78f-f4c16890e9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:34:59.641181 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b8ad7-bf08-48ab-a78f-f4c16890e9ca None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:34:59.641462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b8ad7-bf08-48ab-a78f-f4c16890e9ca None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:34:59.641736 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b8ad7-bf08-48ab-a78f-f4c16890e9ca None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:34:59.642410 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806b8ad7-bf08-48ab-a78f-f4c16890e9ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:34:59.643787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806b8ad7-bf08-48ab-a78f-f4c16890e9ca None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:34:59.650401 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 487/996] 10.222.0.26 () {56 vars in 1087 bytes} [Mon May 9 09:34:59 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:35:00.543182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68321a-6945-422a-abcf-8c04bc6dd580 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:00.543730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68321a-6945-422a-abcf-8c04bc6dd580 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:00.543730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68321a-6945-422a-abcf-8c04bc6dd580 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:00.545517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 510/997] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:00.547234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec712d-4c00-4053-b4f1-66cba01d98c3 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:00.547657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec712d-4c00-4053-b4f1-66cba01d98c3 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:00.547825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec712d-4c00-4053-b4f1-66cba01d98c3 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:00.549479 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 488/998] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:00.550491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c78af6-d714-496d-ab5d-15772c3ea1fc None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:00.550808 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c78af6-d714-496d-ab5d-15772c3ea1fc None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:00.551085 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c78af6-d714-496d-ab5d-15772c3ea1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:00.557603 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8eba6-926a-495a-bc21-a953453e2e3f None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:00.557603 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8eba6-926a-495a-bc21-a953453e2e3f None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:00.557603 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8eba6-926a-495a-bc21-a953453e2e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:00.571030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-f2c78af6-d714-496d-ab5d-15772c3ea1fc None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:00.572228 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-f2c78af6-d714-496d-ab5d-15772c3ea1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:00.572547 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-47c8eba6-926a-495a-bc21-a953453e2e3f None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:00.573727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-47c8eba6-926a-495a-bc21-a953453e2e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:00.636648 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 511/999] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 09:35:00.642532 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 489/1000] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 09:35:00.658204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c78af6-d714-496d-ab5d-15772c3ea1fc None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:00.664495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8eba6-926a-495a-bc21-a953453e2e3f None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:00.753234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c78af6-d714-496d-ab5d-15772c3ea1fc None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:00.763063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8eba6-926a-495a-bc21-a953453e2e3f None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:00.788289 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12b065-2975-495f-8959-728965a7193c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:00.789140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12b065-2975-495f-8959-728965a7193c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:00.789420 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12b065-2975-495f-8959-728965a7193c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:00.789772 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12b065-2975-495f-8959-728965a7193c None admin] PATH_INFO: `/v3/services/image` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:00.790339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12b065-2975-495f-8959-728965a7193c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:00.791732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12b065-2975-495f-8959-728965a7193c None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:00.796588 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a90373-dff2-47fe-92be-de496e33b863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:00.797365 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a90373-dff2-47fe-92be-de496e33b863 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:00.797642 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a90373-dff2-47fe-92be-de496e33b863 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:00.797911 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a90373-dff2-47fe-92be-de496e33b863 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:00.798557 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a90373-dff2-47fe-92be-de496e33b863 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:00.799901 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a90373-dff2-47fe-92be-de496e33b863 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:00.803993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-af12b065-2975-495f-8959-728965a7193c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 09:35:00.803993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:00.803993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:35:00.803993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:35:00.803993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:35:00.803993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:35:00.803993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:35:00.803993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:35:00.803993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 09:35:00.803993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:00.803993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:35:00.803993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:00.803993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:00.803993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:00.803993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:00.803993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:00.803993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:00.810229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:00.810229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:00.810229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:00.810229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:00.810229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:00.810229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:00.810229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:35:00.810229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:35:00.810229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:35:00.810229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:35:00.810229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:00.810229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:00.810229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:00.810229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:00.810229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:00.810229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:00.811104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:00.811104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:35:00.811104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:00.811104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:00.811104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:00.811104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:00.811104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:00.811104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:00.811104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:00.811104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:00.811104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:35:00.811104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:35:00.811104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 09:35:00.811104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:00.811104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 512/1001] 10.222.0.26 () {56 vars in 1105 bytes} [Mon May 9 09:35:00 2022] GET /identity/v3/services/image => generated 86 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:35:00.811910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12b065-2975-495f-8959-728965a7193c None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:00.812344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-d9a90373-dff2-47fe-92be-de496e33b863 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 09:35:00.812344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:00.812344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:35:00.812344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:35:00.812344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:35:00.812344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:35:00.812344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:35:00.812344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:35:00.812344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 09:35:00.812344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:35:00.812344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:35:00.812344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:35:00.812344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:00.812344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:00.812344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:00.812344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:00.812344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:00.814652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:00.814652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:00.814652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:00.814652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:00.814652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:00.814652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:00.814652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:35:00.814652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:35:00.814652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:35:00.814652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:35:00.814652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:00.814652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:00.814652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:00.814652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:00.814652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:00.814652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:00.816980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:00.816980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:35:00.816980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:00.816980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:00.816980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:00.816980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:00.816980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:00.816980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:00.816980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:00.816980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:00.816980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:35:00.816980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:35:00.816980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 09:35:00.816980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:35:00.816980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 490/1002] 10.222.0.26 () {56 vars in 1126 bytes} [Mon May 9 09:35:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:35:00.819292 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a90373-dff2-47fe-92be-de496e33b863 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:00.914576 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a90373-dff2-47fe-92be-de496e33b863 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:00.917543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12b065-2975-495f-8959-728965a7193c None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:00.946535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb870a78-34cf-4a79-ab48-311bbf91cdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:00.947196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb870a78-34cf-4a79-ab48-311bbf91cdf4 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:00.947469 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb870a78-34cf-4a79-ab48-311bbf91cdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:00.947760 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb870a78-34cf-4a79-ab48-311bbf91cdf4 None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:00.948459 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb870a78-34cf-4a79-ab48-311bbf91cdf4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:00.948786 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb870a78-34cf-4a79-ab48-311bbf91cdf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:00.950155 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb870a78-34cf-4a79-ab48-311bbf91cdf4 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:00.954790 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 491/1003] 10.222.0.26 () {56 vars in 1122 bytes} [Mon May 9 09:35:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:35:00.960298 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb870a78-34cf-4a79-ab48-311bbf91cdf4 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:00.960456 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cfa456-09ae-4428-a4d9-98d056387c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:00.961110 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfa456-09ae-4428-a4d9-98d056387c16 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:00.961387 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfa456-09ae-4428-a4d9-98d056387c16 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:00.961655 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfa456-09ae-4428-a4d9-98d056387c16 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:00.962354 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cfa456-09ae-4428-a4d9-98d056387c16 None admin] RBAC: Adding query filter params (name=image) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:00.962655 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cfa456-09ae-4428-a4d9-98d056387c16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:00.964011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cfa456-09ae-4428-a4d9-98d056387c16 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:00.968695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 513/1004] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:35:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:00.973979 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cfa456-09ae-4428-a4d9-98d056387c16 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:01.061553 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb870a78-34cf-4a79-ab48-311bbf91cdf4 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:01.069346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cfa456-09ae-4428-a4d9-98d056387c16 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:01.093371 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8db22-e7e8-4466-9170-c3e221eb5be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:01.094027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8db22-e7e8-4466-9170-c3e221eb5be5 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:01.094299 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8db22-e7e8-4466-9170-c3e221eb5be5 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:01.094549 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8db22-e7e8-4466-9170-c3e221eb5be5 None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:01.095261 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8db22-e7e8-4466-9170-c3e221eb5be5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:01.095556 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8db22-e7e8-4466-9170-c3e221eb5be5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:01.096940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8db22-e7e8-4466-9170-c3e221eb5be5 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:01.102202 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 492/1005] 10.222.0.26 () {56 vars in 1122 bytes} [Mon May 9 09:35:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:35:01.102337 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509d8bd-3b79-4e4d-9867-4f5170ce3fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:01.102702 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509d8bd-3b79-4e4d-9867-4f5170ce3fd1 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:01.102969 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509d8bd-3b79-4e4d-9867-4f5170ce3fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:01.103235 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509d8bd-3b79-4e4d-9867-4f5170ce3fd1 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:01.103925 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509d8bd-3b79-4e4d-9867-4f5170ce3fd1 None admin] RBAC: Adding query filter params (type=image) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:01.104219 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509d8bd-3b79-4e4d-9867-4f5170ce3fd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:01.105622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509d8bd-3b79-4e4d-9867-4f5170ce3fd1 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:01.107662 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8db22-e7e8-4466-9170-c3e221eb5be5 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:01.110525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 514/1006] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:35:00 2022] GET /identity/v3/services?type=image => generated 122 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:01.215316 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8db22-e7e8-4466-9170-c3e221eb5be5 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:01.250821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe096e1-260e-46bf-836b-563b2d826a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:01.251504 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe096e1-260e-46bf-836b-563b2d826a48 None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:01.251751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe096e1-260e-46bf-836b-563b2d826a48 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:01.252043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe096e1-260e-46bf-836b-563b2d826a48 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:01.252820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe096e1-260e-46bf-836b-563b2d826a48 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:01.254220 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe096e1-260e-46bf-836b-563b2d826a48 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:01.277172 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 493/1007] 10.222.0.26 () {60 vars in 1144 bytes} [Mon May 9 09:35:01 2022] POST /identity/v3/endpoints => generated 325 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:35:01.853111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc98e5-8367-478e-9a09-ab021afa2533 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:01.853659 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc98e5-8367-478e-9a09-ab021afa2533 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:01.853659 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc98e5-8367-478e-9a09-ab021afa2533 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:01.855144 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 515/1008] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:01.860714 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310839df-69ed-478a-9b32-3fd99c3b5046 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:01.861145 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310839df-69ed-478a-9b32-3fd99c3b5046 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:01.861317 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310839df-69ed-478a-9b32-3fd99c3b5046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:01.878768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-310839df-69ed-478a-9b32-3fd99c3b5046 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:01.879957 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-310839df-69ed-478a-9b32-3fd99c3b5046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:01.953959 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 494/1009] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:01 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:01.976608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc98e5-8367-478e-9a09-ab021afa2533 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:02.076885 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc98e5-8367-478e-9a09-ab021afa2533 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:02.110565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc52fb-f0dc-44ce-951e-c034070dd017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:02.111285 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc52fb-f0dc-44ce-951e-c034070dd017 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:02.111561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc52fb-f0dc-44ce-951e-c034070dd017 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:02.111827 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc52fb-f0dc-44ce-951e-c034070dd017 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:02.112475 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc52fb-f0dc-44ce-951e-c034070dd017 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:02.113875 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc52fb-f0dc-44ce-951e-c034070dd017 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:02.125859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-13cc52fb-f0dc-44ce-951e-c034070dd017 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 09:35:02.125859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:02.125859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:35:02.125859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:35:02.125859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:35:02.125859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:35:02.125859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:35:02.125859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:35:02.125859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 09:35:02.125859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:02.125859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:35:02.125859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:02.125859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:02.125859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:02.125859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:02.125859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:02.125859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:02.126870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:02.126870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:02.126870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:02.126870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:02.126870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:02.126870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:02.126870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:35:02.126870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:35:02.126870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:35:02.126870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:35:02.126870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:02.126870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:02.126870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:02.126870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:02.126870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:02.126870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:02.127737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:02.127737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:35:02.127737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:02.127737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:02.127737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:02.127737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:02.127737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:02.127737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:02.127737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:02.127737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:02.127737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:35:02.127737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:35:02.127737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 09:35:02.127737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:02.132292 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 516/1010] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:35:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:35:02.133417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310839df-69ed-478a-9b32-3fd99c3b5046 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:02.244193 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310839df-69ed-478a-9b32-3fd99c3b5046 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:02.278728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741788b0-9362-4e26-8164-a7dae472932c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:02.279716 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741788b0-9362-4e26-8164-a7dae472932c None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:02.280210 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741788b0-9362-4e26-8164-a7dae472932c None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:02.280731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741788b0-9362-4e26-8164-a7dae472932c None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:02.281633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741788b0-9362-4e26-8164-a7dae472932c None admin] RBAC: Adding query filter params (name=compute) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:02.282127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741788b0-9362-4e26-8164-a7dae472932c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:02.283741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741788b0-9362-4e26-8164-a7dae472932c None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:02.289211 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 495/1011] 10.222.0.26 () {56 vars in 1112 bytes} [Mon May 9 09:35:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:02.295101 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc52fb-f0dc-44ce-951e-c034070dd017 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:02.394187 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc52fb-f0dc-44ce-951e-c034070dd017 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:02.430047 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d2200-f615-4519-b420-076b137efb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:02.430710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d2200-f615-4519-b420-076b137efb74 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:02.431014 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d2200-f615-4519-b420-076b137efb74 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:02.431250 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d2200-f615-4519-b420-076b137efb74 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:02.432016 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d2200-f615-4519-b420-076b137efb74 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:02.432320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d2200-f615-4519-b420-076b137efb74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:02.433718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d2200-f615-4519-b420-076b137efb74 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:02.438585 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 517/1012] 10.222.0.26 () {56 vars in 1112 bytes} [Mon May 9 09:35:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:02.444240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741788b0-9362-4e26-8164-a7dae472932c None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:02.544564 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741788b0-9362-4e26-8164-a7dae472932c None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:02.581630 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce2a400-19fb-49b6-9403-586b60fb1326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:02.582288 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2a400-19fb-49b6-9403-586b60fb1326 None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:02.582564 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2a400-19fb-49b6-9403-586b60fb1326 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:02.582834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2a400-19fb-49b6-9403-586b60fb1326 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:02.583582 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce2a400-19fb-49b6-9403-586b60fb1326 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:02.584975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce2a400-19fb-49b6-9403-586b60fb1326 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:02.605276 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 496/1013] 10.222.0.26 () {60 vars in 1144 bytes} [Mon May 9 09:35:02 2022] POST /identity/v3/endpoints => generated 334 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:35:03.301903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1ddd5-892b-44bc-ad6d-6fff91e40083 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:03.302470 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1ddd5-892b-44bc-ad6d-6fff91e40083 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:03.302470 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1ddd5-892b-44bc-ad6d-6fff91e40083 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:03.303972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 518/1014] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:03.309411 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98969de-3e5e-4dd8-ba8d-45acf8e23aa8 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:03.309992 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98969de-3e5e-4dd8-ba8d-45acf8e23aa8 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:03.309992 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98969de-3e5e-4dd8-ba8d-45acf8e23aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:03.328003 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-f98969de-3e5e-4dd8-ba8d-45acf8e23aa8 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:03.329232 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f98969de-3e5e-4dd8-ba8d-45acf8e23aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:03.404006 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 497/1015] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:03 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:03.425989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1ddd5-892b-44bc-ad6d-6fff91e40083 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:03.503084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b8ff7-5950-4695-b9b4-aab1a7e85781 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:03.503323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b8ff7-5950-4695-b9b4-aab1a7e85781 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:03.503579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b8ff7-5950-4695-b9b4-aab1a7e85781 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:03.505298 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 498/1016] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:35:03.510338 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94bc89-8dbc-4ef4-95fc-a6d9134f1a07 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:03.510669 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94bc89-8dbc-4ef4-95fc-a6d9134f1a07 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:03.510931 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94bc89-8dbc-4ef4-95fc-a6d9134f1a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:03.527829 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-8a94bc89-8dbc-4ef4-95fc-a6d9134f1a07 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:03.529098 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-8a94bc89-8dbc-4ef4-95fc-a6d9134f1a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:03.531836 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1ddd5-892b-44bc-ad6d-6fff91e40083 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:03.567781 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b3cd8-c383-40a1-9f9a-dd1d8e260759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:03.568568 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b3cd8-c383-40a1-9f9a-dd1d8e260759 None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:03.568780 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b3cd8-c383-40a1-9f9a-dd1d8e260759 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:03.569006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b3cd8-c383-40a1-9f9a-dd1d8e260759 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:03.569728 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b3cd8-c383-40a1-9f9a-dd1d8e260759 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:03.571193 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b3cd8-c383-40a1-9f9a-dd1d8e260759 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:03.586301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 519/1017] 10.222.0.26 () {60 vars in 1141 bytes} [Mon May 9 09:35:03 2022] POST /identity/v3/services => generated 242 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:35:03.625131 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 499/1018] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:03.647949 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b3cd8-c383-40a1-9f9a-dd1d8e260759 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:03.750983 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b3cd8-c383-40a1-9f9a-dd1d8e260759 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:03.786699 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe68ee-2b90-4992-83b4-4652d5ec368f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:03.787494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe68ee-2b90-4992-83b4-4652d5ec368f None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:03.787764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe68ee-2b90-4992-83b4-4652d5ec368f None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:03.788026 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe68ee-2b90-4992-83b4-4652d5ec368f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:03.789316 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe68ee-2b90-4992-83b4-4652d5ec368f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:03.790746 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe68ee-2b90-4992-83b4-4652d5ec368f None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:03.792674 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 520/1019] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:35:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:03.798539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94bc89-8dbc-4ef4-95fc-a6d9134f1a07 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:03.901291 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94bc89-8dbc-4ef4-95fc-a6d9134f1a07 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:03.936182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089bd9b-c897-4cff-b7d8-c7add47b6c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:03.936867 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089bd9b-c897-4cff-b7d8-c7add47b6c4e None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:03.937144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089bd9b-c897-4cff-b7d8-c7add47b6c4e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:03.937416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089bd9b-c897-4cff-b7d8-c7add47b6c4e None admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:03.938121 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089bd9b-c897-4cff-b7d8-c7add47b6c4e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:03.939566 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089bd9b-c897-4cff-b7d8-c7add47b6c4e None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:03.976689 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 500/1020] 10.222.0.26 () {60 vars in 1141 bytes} [Mon May 9 09:35:03 2022] POST /identity/v3/projects => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:35:04.906436 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab52c6-b7dc-4b1e-9e6e-1c5f94c65cf3 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:04.906959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab52c6-b7dc-4b1e-9e6e-1c5f94c65cf3 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:04.906959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab52c6-b7dc-4b1e-9e6e-1c5f94c65cf3 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:04.908210 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 521/1021] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:04.913797 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78741bd-e2e3-4577-9090-fa429e7ac0ca None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:04.914210 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78741bd-e2e3-4577-9090-fa429e7ac0ca None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:04.914379 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78741bd-e2e3-4577-9090-fa429e7ac0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:04.931344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-b78741bd-e2e3-4577-9090-fa429e7ac0ca None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:04.932519 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-b78741bd-e2e3-4577-9090-fa429e7ac0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:05.000819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 501/1022] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:05.021060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab52c6-b7dc-4b1e-9e6e-1c5f94c65cf3 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:05.122203 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab52c6-b7dc-4b1e-9e6e-1c5f94c65cf3 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:05.157023 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a188e2-d439-4645-9826-a9dc727900d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:05.157753 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a188e2-d439-4645-9826-a9dc727900d8 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:05.158031 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a188e2-d439-4645-9826-a9dc727900d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:05.158296 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a188e2-d439-4645-9826-a9dc727900d8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:05.168420 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a188e2-d439-4645-9826-a9dc727900d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:35:05.176494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-d8a188e2-d439-4645-9826-a9dc727900d8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 09:35:05.176494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:05.176494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:05.176494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:05.176494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:05.176494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:05.176494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:05.176494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:05.176494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:05.176494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:05.176494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:05.176494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:05.176494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:35:05.176494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:35:05.176494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:35:05.176494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:35:05.177596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:35:05.177596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:35:05.177596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:35:05.177596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:35:05.177596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:05.177596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:05.177596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:05.177596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:05.177596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:05.177596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:05.177596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:05.177596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:35:05.177596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:05.177596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:05.177596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:05.177596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:05.178481 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:05.178481 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:05.178481 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:05.178481 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:05.178481 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:35:05.178481 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:35:05.178481 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:35:05.178481 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:35:05.178481 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:35:05.178481 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:35:05.178481 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 09:35:05.178481 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:05.181034 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 522/1023] 10.222.0.26 () {56 vars in 1120 bytes} [Mon May 9 09:35:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:35:05.185945 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78741bd-e2e3-4577-9090-fa429e7ac0ca None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:05.287910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78741bd-e2e3-4577-9090-fa429e7ac0ca None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:05.322306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dbae4f-3072-44d9-ad7c-7444ef9a2caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:05.323002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbae4f-3072-44d9-ad7c-7444ef9a2caf None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:05.323279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbae4f-3072-44d9-ad7c-7444ef9a2caf None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:05.323552 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbae4f-3072-44d9-ad7c-7444ef9a2caf None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:05.324302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dbae4f-3072-44d9-ad7c-7444ef9a2caf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:05.324602 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dbae4f-3072-44d9-ad7c-7444ef9a2caf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:05.326009 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dbae4f-3072-44d9-ad7c-7444ef9a2caf None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:05.334760 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 502/1024] 10.222.0.26 () {56 vars in 1115 bytes} [Mon May 9 09:35:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:05.340502 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a188e2-d439-4645-9826-a9dc727900d8 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:05.437704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a188e2-d439-4645-9826-a9dc727900d8 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:05.470817 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a775d-bc83-4a5a-944f-1ca259b4d0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:05.471599 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a775d-bc83-4a5a-944f-1ca259b4d0d9 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:05.471880 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a775d-bc83-4a5a-944f-1ca259b4d0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:05.472149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a775d-bc83-4a5a-944f-1ca259b4d0d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:05.482979 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a775d-bc83-4a5a-944f-1ca259b4d0d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:05.484428 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a775d-bc83-4a5a-944f-1ca259b4d0d9 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:05.502029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-2a7a775d-bc83-4a5a-944f-1ca259b4d0d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:35:05.502029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:05.502029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:05.502029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:05.502029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:05.502029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:05.502029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:05.502029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:05.502029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:05.502029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:05.502029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:05.502029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:05.502029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:35:05.502029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:35:05.502029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:35:05.502978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:35:05.502978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:05.502978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:05.502978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:05.502978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:05.502978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:05.502978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:05.502978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:05.502978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:35:05.502978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:05.502978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:05.502978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:05.502978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:05.502978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:05.502978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:05.502978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:05.502978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:05.503915 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:35:05.503915 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:35:05.503915 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:35:05.503915 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:05.505041 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 523/1025] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:35:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:35:05.509052 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dbae4f-3072-44d9-ad7c-7444ef9a2caf None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:05.611751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dbae4f-3072-44d9-ad7c-7444ef9a2caf None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:05.646960 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8559236-4573-40bb-9791-bd3ba226c26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:05.647644 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8559236-4573-40bb-9791-bd3ba226c26c None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:05.647909 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8559236-4573-40bb-9791-bd3ba226c26c None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:05.648239 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8559236-4573-40bb-9791-bd3ba226c26c None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:05.649020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8559236-4573-40bb-9791-bd3ba226c26c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:05.649328 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8559236-4573-40bb-9791-bd3ba226c26c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:05.650741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8559236-4573-40bb-9791-bd3ba226c26c None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:05.664492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 503/1026] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:05.670012 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a775d-bc83-4a5a-944f-1ca259b4d0d9 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:05.770050 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a775d-bc83-4a5a-944f-1ca259b4d0d9 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:05.802294 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97b63a-9ed7-42dd-a6e5-6bd3a8018720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:05.803095 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97b63a-9ed7-42dd-a6e5-6bd3a8018720 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:05.803373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97b63a-9ed7-42dd-a6e5-6bd3a8018720 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:05.803643 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97b63a-9ed7-42dd-a6e5-6bd3a8018720 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:05.811271 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c97b63a-9ed7-42dd-a6e5-6bd3a8018720 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:05.812749 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c97b63a-9ed7-42dd-a6e5-6bd3a8018720 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:05.828011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-3c97b63a-9ed7-42dd-a6e5-6bd3a8018720 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 09:35:05.828011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:05.828011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:05.828011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:05.828011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:05.828011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:05.828011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:05.828011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:05.828011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:05.828011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:05.828011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:05.828011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:05.828011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:35:05.828011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:35:05.828011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:35:05.828011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:35:05.829409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:05.829409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:05.829409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:35:05.829409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:35:05.829409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:35:05.829409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:35:05.829409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:05.829409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:05.829409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:05.829409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:05.829409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:05.829409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:35:05.829409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:05.829409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:05.829409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:05.829409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:05.829409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:05.829409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:05.830612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:05.830612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:05.830612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:35:05.830612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:35:05.830612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:35:05.830612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:35:05.830612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:35:05.830612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:35:05.830612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 09:35:05.830612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:05.830612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 524/1027] 10.222.0.26 () {56 vars in 1093 bytes} [Mon May 9 09:35:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:35:05.831702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a2e51-d800-4e1c-bb48-628f6e54ca68 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:05.832079 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a2e51-d800-4e1c-bb48-628f6e54ca68 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:05.832763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a2e51-d800-4e1c-bb48-628f6e54ca68 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:05.833359 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97b63a-9ed7-42dd-a6e5-6bd3a8018720 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:05.834383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 504/1028] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:05.839898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5561d-ecf1-4386-8943-6df741ce5c9d None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:05.840147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5561d-ecf1-4386-8943-6df741ce5c9d None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:05.840449 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5561d-ecf1-4386-8943-6df741ce5c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:05.856928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-e7c5561d-ecf1-4386-8943-6df741ce5c9d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:05.858103 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e7c5561d-ecf1-4386-8943-6df741ce5c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:05.923163 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 505/1029] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 09:35:05.928927 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97b63a-9ed7-42dd-a6e5-6bd3a8018720 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:05.945483 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5561d-ecf1-4386-8943-6df741ce5c9d None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:05.961569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7d399-9c01-475b-9597-574c9e37ee0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:05.962286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7d399-9c01-475b-9597-574c9e37ee0a None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:05.962511 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7d399-9c01-475b-9597-574c9e37ee0a None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:05.962763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7d399-9c01-475b-9597-574c9e37ee0a None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:05.963611 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7d399-9c01-475b-9597-574c9e37ee0a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:05.963915 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7d399-9c01-475b-9597-574c9e37ee0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:05.965375 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7d399-9c01-475b-9597-574c9e37ee0a None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:05.986880 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 525/1030] 10.222.0.26 () {56 vars in 1133 bytes} [Mon May 9 09:35:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:35:05.987805 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7d399-9c01-475b-9597-574c9e37ee0a None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:06.044282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5561d-ecf1-4386-8943-6df741ce5c9d None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:06.077018 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86474e4c-b0ef-4a8d-b62d-08b77636973a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:06.077805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86474e4c-b0ef-4a8d-b62d-08b77636973a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:06.078082 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86474e4c-b0ef-4a8d-b62d-08b77636973a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:06.078354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86474e4c-b0ef-4a8d-b62d-08b77636973a None admin] PATH_INFO: `/v3/services/image` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:06.078995 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86474e4c-b0ef-4a8d-b62d-08b77636973a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:06.080375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86474e4c-b0ef-4a8d-b62d-08b77636973a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:06.085840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7d399-9c01-475b-9597-574c9e37ee0a None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:06.091724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-86474e4c-b0ef-4a8d-b62d-08b77636973a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 09:35:06.091724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:06.091724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:35:06.091724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:35:06.091724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:35:06.091724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:35:06.091724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:35:06.091724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:35:06.091724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 09:35:06.091724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:35:06.091724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:35:06.091724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:35:06.091724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:06.091724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:06.091724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:06.091724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:06.091724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:06.092978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:06.092978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:06.092978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:06.092978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:06.092978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:06.092978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:06.092978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:35:06.092978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:35:06.092978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:35:06.092978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:35:06.092978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:06.092978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:06.092978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:06.092978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:06.092978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:06.092978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:06.094226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:06.094226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:35:06.094226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:06.094226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:06.094226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:06.094226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:06.094226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:06.094226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:06.094226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:06.094226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:06.094226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:35:06.094226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:35:06.094226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 09:35:06.094226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:35:06.094226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 506/1031] 10.222.0.26 () {56 vars in 1105 bytes} [Mon May 9 09:35:05 2022] GET /identity/v3/services/image => generated 86 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:35:06.098872 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86474e4c-b0ef-4a8d-b62d-08b77636973a None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:06.122158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db0fb6-7de1-442a-a3a2-021d108c787f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:06.122809 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db0fb6-7de1-442a-a3a2-021d108c787f None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:06.123089 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db0fb6-7de1-442a-a3a2-021d108c787f None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:06.123360 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db0fb6-7de1-442a-a3a2-021d108c787f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:06.185932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75db0fb6-7de1-442a-a3a2-021d108c787f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:06.187736 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75db0fb6-7de1-442a-a3a2-021d108c787f None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:06.193205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86474e4c-b0ef-4a8d-b62d-08b77636973a None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:06.224669 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077634b-7542-42c0-81e7-6409891f67a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:06.225345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077634b-7542-42c0-81e7-6409891f67a1 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:06.225622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077634b-7542-42c0-81e7-6409891f67a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:06.225895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077634b-7542-42c0-81e7-6409891f67a1 None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:06.226650 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077634b-7542-42c0-81e7-6409891f67a1 None admin] RBAC: Adding query filter params (name=image) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:06.226951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077634b-7542-42c0-81e7-6409891f67a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:06.228301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077634b-7542-42c0-81e7-6409891f67a1 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:06.232961 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 507/1032] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:35:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:06.234134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c275a-161a-43e7-b49e-c5f2e236f07f None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:06.234411 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c275a-161a-43e7-b49e-c5f2e236f07f None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:06.234677 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c275a-161a-43e7-b49e-c5f2e236f07f None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:06.236026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 508/1033] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:06.239059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c275a-161a-43e7-b49e-c5f2e236f07f None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:06.281362 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 526/1034] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:35:05 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 09:35:06.282737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ad154-cbe8-4a17-9cdf-e69b1b055959 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:06.283021 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ad154-cbe8-4a17-9cdf-e69b1b055959 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:06.283291 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ad154-cbe8-4a17-9cdf-e69b1b055959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:06.300386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-2f5ad154-cbe8-4a17-9cdf-e69b1b055959 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:06.301536 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-2f5ad154-cbe8-4a17-9cdf-e69b1b055959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:06.336740 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c275a-161a-43e7-b49e-c5f2e236f07f None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:06.364852 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 527/1035] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:06.365665 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ad154-cbe8-4a17-9cdf-e69b1b055959 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:06.380140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3188d-ab7b-472c-812b-5295564aa1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:06.380840 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3188d-ab7b-472c-812b-5295564aa1f6 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:06.381121 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3188d-ab7b-472c-812b-5295564aa1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:06.381388 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3188d-ab7b-472c-812b-5295564aa1f6 None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:06.382090 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3188d-ab7b-472c-812b-5295564aa1f6 None admin] RBAC: Adding query filter params (type=image) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:06.382449 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3188d-ab7b-472c-812b-5295564aa1f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:06.383817 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3188d-ab7b-472c-812b-5295564aa1f6 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:06.388993 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 509/1036] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:35:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:06.389773 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3188d-ab7b-472c-812b-5295564aa1f6 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:06.462185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ad154-cbe8-4a17-9cdf-e69b1b055959 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:06.483489 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3188d-ab7b-472c-812b-5295564aa1f6 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:06.495533 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a655256-28be-4adf-acf6-aa689c9d4315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:06.496326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a655256-28be-4adf-acf6-aa689c9d4315 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:06.496626 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a655256-28be-4adf-acf6-aa689c9d4315 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:06.496898 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a655256-28be-4adf-acf6-aa689c9d4315 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:06.506546 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a655256-28be-4adf-acf6-aa689c9d4315 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:06.508051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a655256-28be-4adf-acf6-aa689c9d4315 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:06.519300 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0813f13-3293-414f-b17b-472dabdb5085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:06.520095 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0813f13-3293-414f-b17b-472dabdb5085 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:06.520373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0813f13-3293-414f-b17b-472dabdb5085 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:06.520681 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0813f13-3293-414f-b17b-472dabdb5085 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:06.522772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0813f13-3293-414f-b17b-472dabdb5085 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:06.524703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0813f13-3293-414f-b17b-472dabdb5085 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:06.526172 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-7a655256-28be-4adf-acf6-aa689c9d4315 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:35:06.526172 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:06.526172 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:06.526172 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:06.526172 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:06.526172 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:06.526172 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:06.526172 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:06.526172 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:06.526172 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:06.526172 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:06.526172 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:06.526172 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:35:06.526172 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:35:06.526172 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:35:06.527242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:35:06.527242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:06.527242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:06.527242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:06.527242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:06.527242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:06.527242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:06.527242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:06.527242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:35:06.527242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:06.527242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:06.527242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:06.527242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:06.527242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:06.527242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:06.527242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:06.527242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:06.534097 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:35:06.534097 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:35:06.534097 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:35:06.534097 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:06.534329 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 510/1037] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:35:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:06.534404 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 528/1038] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:35:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:35:06.534561 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0813f13-3293-414f-b17b-472dabdb5085 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:06.534663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a655256-28be-4adf-acf6-aa689c9d4315 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:06.630917 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a655256-28be-4adf-acf6-aa689c9d4315 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:06.632233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0813f13-3293-414f-b17b-472dabdb5085 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:06.661797 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2fa42a-19ff-425e-8a87-25202652c508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:06.662454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2fa42a-19ff-425e-8a87-25202652c508 None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:06.662732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2fa42a-19ff-425e-8a87-25202652c508 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:06.663003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2fa42a-19ff-425e-8a87-25202652c508 None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:06.663190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a4fa3-be7e-456b-86bd-de663e4ae7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:06.663701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2fa42a-19ff-425e-8a87-25202652c508 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:06.663833 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a4fa3-be7e-456b-86bd-de663e4ae7c6 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:06.664105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a4fa3-be7e-456b-86bd-de663e4ae7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:06.664366 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a4fa3-be7e-456b-86bd-de663e4ae7c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:06.665167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2fa42a-19ff-425e-8a87-25202652c508 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:06.665280 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a4fa3-be7e-456b-86bd-de663e4ae7c6 None admin] RBAC: Adding query filter params (interface=public, service_id=92988b000a424971961440ea20bc36f2, region_id=RegionOne) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:06.665476 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a4fa3-be7e-456b-86bd-de663e4ae7c6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:06.666812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a4fa3-be7e-456b-86bd-de663e4ae7c6 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:06.671725 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 511/1039] 10.222.0.26 () {56 vars in 1251 bytes} [Mon May 9 09:35:06 2022] GET /identity/v3/endpoints?service_id=92988b000a424971961440ea20bc36f2&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:06.672436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a4fa3-be7e-456b-86bd-de663e4ae7c6 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:06.687764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 529/1040] 10.222.0.26 () {60 vars in 1132 bytes} [Mon May 9 09:35:06 2022] POST /identity/v3/users => generated 285 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:35:06.688583 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2fa42a-19ff-425e-8a87-25202652c508 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:06.777794 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a4fa3-be7e-456b-86bd-de663e4ae7c6 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:06.790176 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2fa42a-19ff-425e-8a87-25202652c508 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:06.811784 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95928506-c29d-44d1-9662-602ebf1539bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:06.812435 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95928506-c29d-44d1-9662-602ebf1539bf None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:06.812753 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95928506-c29d-44d1-9662-602ebf1539bf None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:06.813051 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95928506-c29d-44d1-9662-602ebf1539bf None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:06.813788 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95928506-c29d-44d1-9662-602ebf1539bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:06.814094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95928506-c29d-44d1-9662-602ebf1539bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:06.815498 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95928506-c29d-44d1-9662-602ebf1539bf None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:06.823641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c83bc-2053-41a3-8837-83f9a9c07179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:06.824319 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c83bc-2053-41a3-8837-83f9a9c07179 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:06.824594 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c83bc-2053-41a3-8837-83f9a9c07179 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:06.824891 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c83bc-2053-41a3-8837-83f9a9c07179 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:06.825622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c83bc-2053-41a3-8837-83f9a9c07179 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:06.827110 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c83bc-2053-41a3-8837-83f9a9c07179 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:06.829344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 512/1041] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:06.834352 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 530/1042] 10.222.0.26 () {56 vars in 1087 bytes} [Mon May 9 09:35:06 2022] GET /identity/v3/services => generated 1957 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:35:06.834765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95928506-c29d-44d1-9662-602ebf1539bf None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:06.947146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95928506-c29d-44d1-9662-602ebf1539bf None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:06.983965 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b171c9b-9dce-45ad-b2e4-3c59da8ee631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:06.984802 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b171c9b-9dce-45ad-b2e4-3c59da8ee631 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:06.985082 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b171c9b-9dce-45ad-b2e4-3c59da8ee631 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:06.985351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b171c9b-9dce-45ad-b2e4-3c59da8ee631 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:06.992796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b171c9b-9dce-45ad-b2e4-3c59da8ee631 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:06.994286 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b171c9b-9dce-45ad-b2e4-3c59da8ee631 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:07.009311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-7b171c9b-9dce-45ad-b2e4-3c59da8ee631 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:35:07.009311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:07.009311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:07.009311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:07.009311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:07.009311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:07.009311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:07.009311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:07.009311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:07.009311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:07.009311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:07.009311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:07.009311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:35:07.009311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:35:07.009311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:35:07.010397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:35:07.010397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:07.010397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:07.010397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:07.010397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:07.010397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:07.010397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:07.010397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:07.010397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:35:07.010397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:07.010397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:07.010397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:07.010397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:07.010397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:07.010397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:07.010397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:07.010397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:07.011286 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:35:07.011286 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:35:07.011286 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:35:07.011286 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:35:07.011286 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:35:07.011286 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:35:07.011286 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:35:07.011286 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:35:07.011286 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 513/1043] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:35:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:35:07.016293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c83bc-2053-41a3-8837-83f9a9c07179 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:07.112003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c83bc-2053-41a3-8837-83f9a9c07179 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:07.145604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d59e45-36e9-40e3-b514-d57214516f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:07.146262 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d59e45-36e9-40e3-b514-d57214516f7b None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:07.146544 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d59e45-36e9-40e3-b514-d57214516f7b None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:07.146812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d59e45-36e9-40e3-b514-d57214516f7b None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:07.147599 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d59e45-36e9-40e3-b514-d57214516f7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:07.147898 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d59e45-36e9-40e3-b514-d57214516f7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:07.149397 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d59e45-36e9-40e3-b514-d57214516f7b None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:07.161554 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 531/1044] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:35:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:07.167410 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b171c9b-9dce-45ad-b2e4-3c59da8ee631 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:07.267293 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b171c9b-9dce-45ad-b2e4-3c59da8ee631 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:07.299390 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136db38-6f1c-4713-8b09-e2a4dd683f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:07.300043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136db38-6f1c-4713-8b09-e2a4dd683f8e None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:07.300317 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136db38-6f1c-4713-8b09-e2a4dd683f8e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:07.300582 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136db38-6f1c-4713-8b09-e2a4dd683f8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:07.301448 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136db38-6f1c-4713-8b09-e2a4dd683f8e None admin] RBAC: Adding query filter params (role.id=e8b800faebb04b2eb2cebfb7a856d7ad, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=6362303cb918486c8525225914332ad5) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:07.301752 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136db38-6f1c-4713-8b09-e2a4dd683f8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:07.303171 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136db38-6f1c-4713-8b09-e2a4dd683f8e None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:07.308935 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 514/1045] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:35:07 2022] GET /identity/v3/role_assignments?user.id=6362303cb918486c8525225914332ad5&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=e8b800faebb04b2eb2cebfb7a856d7ad => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:08.941295 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd3d49-7a02-4458-b608-3861e6d6cd30 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:08.941819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd3d49-7a02-4458-b608-3861e6d6cd30 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:08.941819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd3d49-7a02-4458-b608-3861e6d6cd30 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:08.943253 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 532/1046] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:08.948475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376203e-c378-492a-8988-de0f61f34d8d None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:08.949353 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376203e-c378-492a-8988-de0f61f34d8d None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:08.949497 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376203e-c378-492a-8988-de0f61f34d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:08.966994 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-2376203e-c378-492a-8988-de0f61f34d8d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:08.968181 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-2376203e-c378-492a-8988-de0f61f34d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:09.033363 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761b294-f9d9-4185-a1ab-faba72ac2298 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:09.033750 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761b294-f9d9-4185-a1ab-faba72ac2298 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:09.034102 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761b294-f9d9-4185-a1ab-faba72ac2298 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:09.035547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 533/1047] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:09.040460 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 515/1048] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:09.040587 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b867c5b-7ba0-434f-b1b7-e4bdd0175a34 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:09.040796 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b867c5b-7ba0-434f-b1b7-e4bdd0175a34 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:09.041062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b867c5b-7ba0-434f-b1b7-e4bdd0175a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:09.059356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-8b867c5b-7ba0-434f-b1b7-e4bdd0175a34 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:09.060914 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-8b867c5b-7ba0-434f-b1b7-e4bdd0175a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:09.063426 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376203e-c378-492a-8988-de0f61f34d8d None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:09.128705 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 534/1049] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 09:35:09.149913 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b867c5b-7ba0-434f-b1b7-e4bdd0175a34 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:09.160613 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376203e-c378-492a-8988-de0f61f34d8d None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:09.194232 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e1223-d230-48ec-abe2-669229295812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:09.195034 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e1223-d230-48ec-abe2-669229295812 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:09.195309 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e1223-d230-48ec-abe2-669229295812 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:09.195573 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e1223-d230-48ec-abe2-669229295812 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:09.205821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e1223-d230-48ec-abe2-669229295812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:35:09.213393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-248e1223-d230-48ec-abe2-669229295812 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:35:09.213393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:09.213393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:09.213393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:09.213393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:09.213393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:09.213393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:09.213393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:09.213393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:09.213393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:09.213393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:09.213393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:09.213393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:35:09.213393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:35:09.213393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:35:09.213393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:35:09.214428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:35:09.214428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:35:09.214428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:35:09.214428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:35:09.214428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:09.214428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:09.214428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:09.214428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:09.214428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:09.214428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:09.214428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:09.214428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:35:09.214428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:09.214428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:09.214428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:09.214428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:09.215279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:09.215279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:09.215279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:09.215279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:09.215279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:35:09.215279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:35:09.215279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:35:09.215279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:35:09.215279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:35:09.215279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:35:09.215279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:35:09.215279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:35:09.215279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 516/1050] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:35:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:35:09.220329 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e1223-d230-48ec-abe2-669229295812 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:09.245157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b867c5b-7ba0-434f-b1b7-e4bdd0175a34 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:09.278149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aef398-2ef4-4571-a68c-dc13df1e4c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:09.278937 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aef398-2ef4-4571-a68c-dc13df1e4c22 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:09.279213 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aef398-2ef4-4571-a68c-dc13df1e4c22 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:09.279483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aef398-2ef4-4571-a68c-dc13df1e4c22 None admin] PATH_INFO: `/v3/services/image` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:09.280127 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aef398-2ef4-4571-a68c-dc13df1e4c22 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:09.281528 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aef398-2ef4-4571-a68c-dc13df1e4c22 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:09.292700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-b9aef398-2ef4-4571-a68c-dc13df1e4c22 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 09:35:09.292700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:09.292700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:35:09.292700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:35:09.292700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:35:09.292700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:35:09.292700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:35:09.292700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:35:09.292700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 09:35:09.292700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:09.292700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:35:09.292700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:09.292700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:09.292700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:09.292700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:09.292700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:09.292700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:09.293723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:09.293723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:09.293723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:09.293723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:09.293723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:09.293723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:09.293723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:35:09.293723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:35:09.293723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:35:09.293723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:35:09.293723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:09.293723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:09.293723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:09.293723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:09.293723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:09.293723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:09.294597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:09.294597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:35:09.294597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:09.294597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:09.294597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:09.294597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:09.294597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:09.294597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:09.294597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:09.294597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:09.294597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:35:09.294597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:35:09.294597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 09:35:09.294597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:09.294597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 535/1051] 10.222.0.26 () {56 vars in 1105 bytes} [Mon May 9 09:35:09 2022] GET /identity/v3/services/image => generated 86 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:35:09.299842 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aef398-2ef4-4571-a68c-dc13df1e4c22 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:09.322070 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e1223-d230-48ec-abe2-669229295812 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:09.356137 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b3c3f-31b0-48db-b8b2-07658d2cdf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:09.356829 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b3c3f-31b0-48db-b8b2-07658d2cdf7d None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:09.357108 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b3c3f-31b0-48db-b8b2-07658d2cdf7d None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:09.357372 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b3c3f-31b0-48db-b8b2-07658d2cdf7d None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:09.358109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b3c3f-31b0-48db-b8b2-07658d2cdf7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:09.358407 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b3c3f-31b0-48db-b8b2-07658d2cdf7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:09.359768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b3c3f-31b0-48db-b8b2-07658d2cdf7d None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:09.368315 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 517/1052] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:35:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:35:09.374127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b3c3f-31b0-48db-b8b2-07658d2cdf7d None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:09.396641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aef398-2ef4-4571-a68c-dc13df1e4c22 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:09.430733 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606acd9-6adf-4c10-aede-09cb7bf953e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:09.431618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606acd9-6adf-4c10-aede-09cb7bf953e2 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:09.432034 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606acd9-6adf-4c10-aede-09cb7bf953e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:09.432461 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606acd9-6adf-4c10-aede-09cb7bf953e2 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:09.433347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606acd9-6adf-4c10-aede-09cb7bf953e2 None admin] RBAC: Adding query filter params (name=image) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:09.433789 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606acd9-6adf-4c10-aede-09cb7bf953e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:09.435290 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606acd9-6adf-4c10-aede-09cb7bf953e2 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:09.440376 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 536/1053] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:35:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:35:09.446381 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606acd9-6adf-4c10-aede-09cb7bf953e2 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:09.473649 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b3c3f-31b0-48db-b8b2-07658d2cdf7d None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:09.510277 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aca910-385a-4008-a7d4-465133747ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:09.511151 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aca910-385a-4008-a7d4-465133747ee2 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:09.511428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aca910-385a-4008-a7d4-465133747ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:09.511702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aca910-385a-4008-a7d4-465133747ee2 None admin] PATH_INFO: `/v3/users/1044bb831aac48fe84c33327e3646227` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:09.524654 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aca910-385a-4008-a7d4-465133747ee2 None admin] RBAC: Authorizing `identity:get_user(user_id=1044bb831aac48fe84c33327e3646227)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:09.526139 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aca910-385a-4008-a7d4-465133747ee2 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:09.527831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 518/1054] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:35:09 2022] GET /identity/v3/users/1044bb831aac48fe84c33327e3646227 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:09.529076 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61832434-ea94-4447-b445-5c21721d0d22 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:09.529356 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61832434-ea94-4447-b445-5c21721d0d22 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:09.529623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61832434-ea94-4447-b445-5c21721d0d22 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:09.531008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 519/1055] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:09.533483 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61832434-ea94-4447-b445-5c21721d0d22 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:09.543752 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606acd9-6adf-4c10-aede-09cb7bf953e2 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:09.580474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b090dc8-c89a-4176-b3b5-79862c0cc80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:09.581165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b090dc8-c89a-4176-b3b5-79862c0cc80d None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:09.581438 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b090dc8-c89a-4176-b3b5-79862c0cc80d None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:09.581703 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b090dc8-c89a-4176-b3b5-79862c0cc80d None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:09.582402 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b090dc8-c89a-4176-b3b5-79862c0cc80d None admin] RBAC: Adding query filter params (type=image) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:09.582696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b090dc8-c89a-4176-b3b5-79862c0cc80d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:09.584058 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b090dc8-c89a-4176-b3b5-79862c0cc80d None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:09.588756 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 537/1056] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:35:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:35:09.589857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aefe8d-0714-41b4-bcd8-04bce434a23d None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:09.590158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aefe8d-0714-41b4-bcd8-04bce434a23d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:09.590398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aefe8d-0714-41b4-bcd8-04bce434a23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:09.605624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-a4aefe8d-0714-41b4-bcd8-04bce434a23d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:09.606807 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-a4aefe8d-0714-41b4-bcd8-04bce434a23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:09.633189 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61832434-ea94-4447-b445-5c21721d0d22 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:09.667206 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49ccea-3a8c-4333-b464-b9ce27c859d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:09.667843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49ccea-3a8c-4333-b464-b9ce27c859d8 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:09.668120 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49ccea-3a8c-4333-b464-b9ce27c859d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:09.668390 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49ccea-3a8c-4333-b464-b9ce27c859d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:09.668904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 538/1057] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:09.673540 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aefe8d-0714-41b4-bcd8-04bce434a23d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:09.737027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e49ccea-3a8c-4333-b464-b9ce27c859d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:09.738382 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e49ccea-3a8c-4333-b464-b9ce27c859d8 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:09.769501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aefe8d-0714-41b4-bcd8-04bce434a23d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:09.800300 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10441425-1a5b-4f21-a8e8-d33927923cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:09.801077 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10441425-1a5b-4f21-a8e8-d33927923cdd None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:09.801359 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10441425-1a5b-4f21-a8e8-d33927923cdd None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:09.801630 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10441425-1a5b-4f21-a8e8-d33927923cdd None admin] PATH_INFO: `/v3/endpoints` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:09.802371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10441425-1a5b-4f21-a8e8-d33927923cdd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:09.803726 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10441425-1a5b-4f21-a8e8-d33927923cdd None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:09.823765 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 539/1058] 10.222.0.26 () {60 vars in 1144 bytes} [Mon May 9 09:35:09 2022] POST /identity/v3/endpoints => generated 327 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:35:09.824572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10441425-1a5b-4f21-a8e8-d33927923cdd None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:09.833880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 520/1059] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:35:09 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 09:35:09.840093 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49ccea-3a8c-4333-b464-b9ce27c859d8 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:09.937074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10441425-1a5b-4f21-a8e8-d33927923cdd None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:09.946831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49ccea-3a8c-4333-b464-b9ce27c859d8 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:09.971107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eef8d9-8c14-4da5-aa06-739888e1c5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:09.971925 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eef8d9-8c14-4da5-aa06-739888e1c5ab None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:09.972203 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eef8d9-8c14-4da5-aa06-739888e1c5ab None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:09.972463 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eef8d9-8c14-4da5-aa06-739888e1c5ab None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:09.981690 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638da65-dec0-49ed-af08-9f718f5e0b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:09.982554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638da65-dec0-49ed-af08-9f718f5e0b9b None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:09.982822 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638da65-dec0-49ed-af08-9f718f5e0b9b None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:09.983101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638da65-dec0-49ed-af08-9f718f5e0b9b None admin] PATH_INFO: `/v3/projects/7725886bec4343b9ade980294d6b0c88` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:09.983480 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eef8d9-8c14-4da5-aa06-739888e1c5ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:35:09.984347 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638da65-dec0-49ed-af08-9f718f5e0b9b None admin] RBAC: Authorizing `identity:get_project(project_id=7725886bec4343b9ade980294d6b0c88)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:09.985849 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638da65-dec0-49ed-af08-9f718f5e0b9b None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:09.987848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 521/1060] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:35:09 2022] GET /identity/v3/projects/7725886bec4343b9ade980294d6b0c88 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:09.991933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-89eef8d9-8c14-4da5-aa06-739888e1c5ab None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 09:35:09.991933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:09.991933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:09.991933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:09.991933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:09.991933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:09.991933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:09.991933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:09.991933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:09.991933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:09.991933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:09.991933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:09.991933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:35:09.991933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:35:09.991933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:35:09.991933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:35:09.993255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:35:09.993255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:35:09.993255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:35:09.993255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:35:09.993255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:09.993255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:09.993255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:09.993255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:09.993255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:09.993255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:09.993255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:09.993255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:35:09.993255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:09.993255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:09.993255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:09.993255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:09.994140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:09.994140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:09.994140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:09.994140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:09.994140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:35:09.994140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:35:09.994140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:35:09.994140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:35:09.994140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:35:09.994140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:35:09.994140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 09:35:09.994140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:09.994140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 540/1061] 10.222.0.26 () {56 vars in 1120 bytes} [Mon May 9 09:35:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:35:09.995185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638da65-dec0-49ed-af08-9f718f5e0b9b None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:09.999406 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eef8d9-8c14-4da5-aa06-739888e1c5ab None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:10.096781 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638da65-dec0-49ed-af08-9f718f5e0b9b None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:10.099702 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eef8d9-8c14-4da5-aa06-739888e1c5ab None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:10.131107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9b798-53a6-4442-9a8a-e39cf91e012c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:10.132104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9b798-53a6-4442-9a8a-e39cf91e012c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:10.132593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9b798-53a6-4442-9a8a-e39cf91e012c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:10.132891 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9b798-53a6-4442-9a8a-e39cf91e012c None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:10.133635 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a9b798-53a6-4442-9a8a-e39cf91e012c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:10.133934 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a9b798-53a6-4442-9a8a-e39cf91e012c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:10.134467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82a155-eef2-4187-b64b-8dfe1b0ce31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:10.135101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82a155-eef2-4187-b64b-8dfe1b0ce31e None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:10.135296 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a9b798-53a6-4442-9a8a-e39cf91e012c None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:10.135401 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82a155-eef2-4187-b64b-8dfe1b0ce31e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:10.135634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82a155-eef2-4187-b64b-8dfe1b0ce31e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:10.136460 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a82a155-eef2-4187-b64b-8dfe1b0ce31e None admin] RBAC: Adding query filter params (role.id=70ac549382b14b2cbb8b35e5dfc648d4, scope.project.id=7725886bec4343b9ade980294d6b0c88, user.id=1044bb831aac48fe84c33327e3646227) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:10.136971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a82a155-eef2-4187-b64b-8dfe1b0ce31e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:10.138741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a82a155-eef2-4187-b64b-8dfe1b0ce31e None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:10.143434 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 541/1062] 10.222.0.26 () {56 vars in 1115 bytes} [Mon May 9 09:35:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:35:10.156962 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 522/1063] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:35:09 2022] GET /identity/v3/role_assignments?user.id=1044bb831aac48fe84c33327e3646227&scope.project.id=7725886bec4343b9ade980294d6b0c88&role.id=70ac549382b14b2cbb8b35e5dfc648d4 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:10.157211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9b798-53a6-4442-9a8a-e39cf91e012c None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:10.254904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9b798-53a6-4442-9a8a-e39cf91e012c None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:10.290706 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b87b14-e649-450f-afb5-e161763cc993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:10.291528 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b87b14-e649-450f-afb5-e161763cc993 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:10.291789 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b87b14-e649-450f-afb5-e161763cc993 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:10.292054 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b87b14-e649-450f-afb5-e161763cc993 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:10.304717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b87b14-e649-450f-afb5-e161763cc993 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:10.306222 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b87b14-e649-450f-afb5-e161763cc993 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:10.324446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-02b87b14-e649-450f-afb5-e161763cc993 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:35:10.324446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:10.324446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:10.324446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:10.324446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:10.324446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:10.324446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:10.324446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:10.324446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:10.324446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:10.324446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:10.324446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:10.324446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:35:10.324446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:35:10.324446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:35:10.325710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:35:10.325710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:10.325710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:10.325710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:10.325710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:10.325710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:10.325710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:10.325710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:10.325710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:35:10.325710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:10.325710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:10.325710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:10.325710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:10.325710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:10.325710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:10.325710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:10.325710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:10.326610 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:35:10.326610 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:35:10.326610 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:35:10.326610 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:10.328757 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 542/1064] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:35:10.331484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82a155-eef2-4187-b64b-8dfe1b0ce31e None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:10.429735 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82a155-eef2-4187-b64b-8dfe1b0ce31e None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:10.464070 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2be545-b7ba-4ca0-95dd-fafc030268b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:10.464763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2be545-b7ba-4ca0-95dd-fafc030268b5 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:10.465042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2be545-b7ba-4ca0-95dd-fafc030268b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:10.465312 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2be545-b7ba-4ca0-95dd-fafc030268b5 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:10.466041 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2be545-b7ba-4ca0-95dd-fafc030268b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:10.466343 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2be545-b7ba-4ca0-95dd-fafc030268b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:10.467715 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2be545-b7ba-4ca0-95dd-fafc030268b5 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:10.479651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 523/1065] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:10.485514 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b87b14-e649-450f-afb5-e161763cc993 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:10.589413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b87b14-e649-450f-afb5-e161763cc993 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:10.623359 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d361a-a859-42cb-99d4-89b0db2f05b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:10.624151 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d361a-a859-42cb-99d4-89b0db2f05b4 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:10.624426 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d361a-a859-42cb-99d4-89b0db2f05b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:10.624706 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d361a-a859-42cb-99d4-89b0db2f05b4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:10.632265 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d361a-a859-42cb-99d4-89b0db2f05b4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:10.633749 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d361a-a859-42cb-99d4-89b0db2f05b4 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:10.647615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-4b7d361a-a859-42cb-99d4-89b0db2f05b4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 09:35:10.647615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:10.647615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:10.647615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:10.647615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:10.647615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:10.647615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:10.647615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:10.647615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:10.647615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:10.647615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:10.647615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:10.647615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:35:10.647615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:35:10.647615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:35:10.647615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:35:10.648695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:10.648695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:10.648695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:35:10.648695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:35:10.648695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:35:10.648695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:35:10.648695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:10.648695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:10.648695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:10.648695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:10.648695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:10.648695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:35:10.648695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:10.648695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:10.648695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:10.648695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:10.648695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:10.648695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:10.649679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:10.649679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:10.649679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:35:10.649679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:35:10.649679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:35:10.649679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:35:10.649679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:35:10.649679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:35:10.649679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 09:35:10.649679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:10.649679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 543/1066] 10.222.0.26 () {56 vars in 1093 bytes} [Mon May 9 09:35:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:35:10.654878 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2be545-b7ba-4ca0-95dd-fafc030268b5 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:10.756497 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2be545-b7ba-4ca0-95dd-fafc030268b5 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:10.789814 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880a36f-4090-4e1c-a9aa-f6dc7a21430b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:10.790477 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880a36f-4090-4e1c-a9aa-f6dc7a21430b None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:10.790740 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880a36f-4090-4e1c-a9aa-f6dc7a21430b None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:10.791010 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880a36f-4090-4e1c-a9aa-f6dc7a21430b None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:10.791842 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880a36f-4090-4e1c-a9aa-f6dc7a21430b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:10.792148 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880a36f-4090-4e1c-a9aa-f6dc7a21430b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:10.793623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880a36f-4090-4e1c-a9aa-f6dc7a21430b None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:10.810688 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 524/1067] 10.222.0.26 () {56 vars in 1133 bytes} [Mon May 9 09:35:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:10.816232 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d361a-a859-42cb-99d4-89b0db2f05b4 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:10.912839 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d361a-a859-42cb-99d4-89b0db2f05b4 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:10.946039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d1791-1e75-41b4-ba3f-7ec37a4ea7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:10.946836 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d1791-1e75-41b4-ba3f-7ec37a4ea7ee None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:10.947117 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d1791-1e75-41b4-ba3f-7ec37a4ea7ee None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:10.947387 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d1791-1e75-41b4-ba3f-7ec37a4ea7ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:10.957833 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d1791-1e75-41b4-ba3f-7ec37a4ea7ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:10.959306 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d1791-1e75-41b4-ba3f-7ec37a4ea7ee None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:10.977541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-3d4d1791-1e75-41b4-ba3f-7ec37a4ea7ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:35:10.977541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:10.977541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:10.977541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:10.977541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:10.977541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:10.977541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:10.977541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:10.977541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:10.977541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:10.977541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:10.977541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:10.977541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:35:10.977541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:35:10.977541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:35:10.978989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:35:10.978989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:10.978989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:10.978989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:10.978989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:10.978989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:10.978989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:10.978989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:10.978989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:35:10.978989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:10.978989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:10.978989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:10.978989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:10.978989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:10.978989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:10.978989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:10.978989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:10.979896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:35:10.979896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:35:10.979896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:35:10.979896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:10.981052 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 544/1068] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:35:10.984814 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880a36f-4090-4e1c-a9aa-f6dc7a21430b None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:11.079672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880a36f-4090-4e1c-a9aa-f6dc7a21430b None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:11.112792 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13805a71-dfe3-4a0e-8909-6c81d0f395e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:11.113447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13805a71-dfe3-4a0e-8909-6c81d0f395e4 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:11.113716 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13805a71-dfe3-4a0e-8909-6c81d0f395e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:11.113980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13805a71-dfe3-4a0e-8909-6c81d0f395e4 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:11.114691 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13805a71-dfe3-4a0e-8909-6c81d0f395e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:11.114985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13805a71-dfe3-4a0e-8909-6c81d0f395e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:11.116346 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13805a71-dfe3-4a0e-8909-6c81d0f395e4 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:11.127655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 525/1069] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:11.133109 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d1791-1e75-41b4-ba3f-7ec37a4ea7ee None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:11.228225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d1791-1e75-41b4-ba3f-7ec37a4ea7ee None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:11.262438 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5a8f0-8d32-451b-8b36-0544c2a701e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:11.263633 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5a8f0-8d32-451b-8b36-0544c2a701e6 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:11.264116 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5a8f0-8d32-451b-8b36-0544c2a701e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:11.264627 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5a8f0-8d32-451b-8b36-0544c2a701e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:11.271203 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5a8f0-8d32-451b-8b36-0544c2a701e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:11.272889 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5a8f0-8d32-451b-8b36-0544c2a701e6 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:11.286820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-a7b5a8f0-8d32-451b-8b36-0544c2a701e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:35:11.286820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:11.286820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:11.286820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:11.286820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:11.286820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:11.286820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:11.286820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:11.286820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:11.286820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:11.286820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:11.286820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:11.286820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:35:11.286820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:35:11.286820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:35:11.287837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:35:11.287837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:11.287837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:11.287837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:11.287837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:11.287837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:11.287837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:11.287837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:11.287837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:35:11.287837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:11.287837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:11.287837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:11.287837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:11.287837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:11.287837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:11.287837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:11.287837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:11.288848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:35:11.288848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:35:11.288848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:35:11.288848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:35:11.288848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:35:11.288848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:35:11.288848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:35:11.288848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:11.292808 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 545/1070] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:35:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:35:11.294716 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13805a71-dfe3-4a0e-8909-6c81d0f395e4 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:11.393876 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13805a71-dfe3-4a0e-8909-6c81d0f395e4 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:11.426002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f9393-66a2-4986-9c25-4e9765eace34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:11.426893 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f9393-66a2-4986-9c25-4e9765eace34 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:11.427345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f9393-66a2-4986-9c25-4e9765eace34 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:11.427776 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f9393-66a2-4986-9c25-4e9765eace34 None admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:11.428743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f9393-66a2-4986-9c25-4e9765eace34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:11.429224 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f9393-66a2-4986-9c25-4e9765eace34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:11.430799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f9393-66a2-4986-9c25-4e9765eace34 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:11.443029 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 526/1071] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:35:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:11.449283 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5a8f0-8d32-451b-8b36-0544c2a701e6 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:11.562051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5a8f0-8d32-451b-8b36-0544c2a701e6 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:11.596446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05101b8-7255-4d67-9b73-8bc9583645a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:11.597383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05101b8-7255-4d67-9b73-8bc9583645a9 None admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:11.597666 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05101b8-7255-4d67-9b73-8bc9583645a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:11.597938 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05101b8-7255-4d67-9b73-8bc9583645a9 None admin] PATH_INFO: `/v3/projects/9cd39464055141b8b43fde20d0885c99/users/6362303cb918486c8525225914332ad5/roles/e8b800faebb04b2eb2cebfb7a856d7ad` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:11.600183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05101b8-7255-4d67-9b73-8bc9583645a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd39464055141b8b43fde20d0885c99, user_id=6362303cb918486c8525225914332ad5, role_id=e8b800faebb04b2eb2cebfb7a856d7ad)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:11.601764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05101b8-7255-4d67-9b73-8bc9583645a9 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:11.614927 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 546/1072] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:35:11 2022] PUT /identity/v3/projects/9cd39464055141b8b43fde20d0885c99/users/6362303cb918486c8525225914332ad5/roles/e8b800faebb04b2eb2cebfb7a856d7ad => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:35:12.075807 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7505b75-12d6-4e70-a1c1-b5bdde4039d6 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:12.076359 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7505b75-12d6-4e70-a1c1-b5bdde4039d6 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:12.076359 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7505b75-12d6-4e70-a1c1-b5bdde4039d6 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:12.077741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 527/1073] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:12.083327 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4f3c2-a408-4096-bb59-02e7e6fa2878 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:12.083783 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4f3c2-a408-4096-bb59-02e7e6fa2878 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:12.083867 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4f3c2-a408-4096-bb59-02e7e6fa2878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:12.101638 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-aeb4f3c2-a408-4096-bb59-02e7e6fa2878 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:12.102834 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-aeb4f3c2-a408-4096-bb59-02e7e6fa2878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:12.174412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 547/1074] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:12.196887 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7505b75-12d6-4e70-a1c1-b5bdde4039d6 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:12.299390 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7505b75-12d6-4e70-a1c1-b5bdde4039d6 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:12.332287 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d06718-827b-4304-8e4e-6ab415507db8 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:12.332471 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d06718-827b-4304-8e4e-6ab415507db8 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:12.332735 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d06718-827b-4304-8e4e-6ab415507db8 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:12.333866 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c41d4-a362-4397-8dd7-cb71a52a2421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:12.334365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 548/1075] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:35:12.334488 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c41d4-a362-4397-8dd7-cb71a52a2421 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:12.334779 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c41d4-a362-4397-8dd7-cb71a52a2421 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:12.335014 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c41d4-a362-4397-8dd7-cb71a52a2421 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:12.339167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c69fc4-074b-46e6-ad8b-c8ddcfcdbc34 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:12.339444 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c69fc4-074b-46e6-ad8b-c8ddcfcdbc34 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:12.339708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c69fc4-074b-46e6-ad8b-c8ddcfcdbc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:12.355887 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-70c69fc4-074b-46e6-ad8b-c8ddcfcdbc34 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:12.357094 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-70c69fc4-074b-46e6-ad8b-c8ddcfcdbc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:12.413132 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c41d4-a362-4397-8dd7-cb71a52a2421 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:12.415399 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c41d4-a362-4397-8dd7-cb71a52a2421 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:12.421576 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 549/1076] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 09:35:12.443687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c69fc4-074b-46e6-ad8b-c8ddcfcdbc34 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:12.527184 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 528/1077] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:35:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 09:35:12.528567 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c41d4-a362-4397-8dd7-cb71a52a2421 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:12.547489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c69fc4-074b-46e6-ad8b-c8ddcfcdbc34 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:12.583785 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851322b-f9a1-4e47-93d2-0533268f1d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:12.584505 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851322b-f9a1-4e47-93d2-0533268f1d10 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:12.584810 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851322b-f9a1-4e47-93d2-0533268f1d10 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:12.585077 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851322b-f9a1-4e47-93d2-0533268f1d10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:12.594577 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9851322b-f9a1-4e47-93d2-0533268f1d10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:35:12.602001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-9851322b-f9a1-4e47-93d2-0533268f1d10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:35:12.602001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:12.602001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:12.602001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:12.602001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:12.602001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:12.602001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:12.602001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:12.602001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:12.602001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:12.602001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:12.602001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:12.602001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:35:12.602001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:35:12.602001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:35:12.602001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:35:12.603185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:35:12.603185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:35:12.603185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:35:12.603185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:35:12.603185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:12.603185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:12.603185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:12.603185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:12.603185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:12.603185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:12.603185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:12.603185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:35:12.603185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:12.603185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:12.603185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:12.603185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:12.604149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:12.604149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:12.604149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:12.604149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:12.604149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:35:12.604149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:35:12.604149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:35:12.604149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:35:12.604149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:35:12.604149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:35:12.604149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:35:12.604149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:12.604149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 550/1078] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:35:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:35:12.608999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851322b-f9a1-4e47-93d2-0533268f1d10 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:12.635535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c41d4-a362-4397-8dd7-cb71a52a2421 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:12.667933 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45c88e-90f5-462b-b750-fd5d3f0287ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:12.668742 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45c88e-90f5-462b-b750-fd5d3f0287ef None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:12.669025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45c88e-90f5-462b-b750-fd5d3f0287ef None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:12.669293 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45c88e-90f5-462b-b750-fd5d3f0287ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:12.679089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45c88e-90f5-462b-b750-fd5d3f0287ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:12.680531 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45c88e-90f5-462b-b750-fd5d3f0287ef None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:12.698008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-5c45c88e-90f5-462b-b750-fd5d3f0287ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:35:12.698008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:12.698008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:12.698008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:12.698008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:12.698008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:12.698008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:12.698008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:12.698008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:12.698008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:12.698008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:12.698008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:12.698008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:35:12.698008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:35:12.698008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:35:12.698991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:35:12.698991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:12.698991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:12.698991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:12.698991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:12.698991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:12.698991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:12.698991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:12.698991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:35:12.698991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:12.698991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:12.698991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:12.698991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:12.698991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:12.698991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:12.698991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:12.698991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:12.699910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:35:12.699910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:35:12.699910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:35:12.699910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:35:12.699910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 529/1079] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:35:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:35:12.704950 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45c88e-90f5-462b-b750-fd5d3f0287ef None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:12.709744 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851322b-f9a1-4e47-93d2-0533268f1d10 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:12.743147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c369bc-3738-4623-a40d-77046a500b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:12.743819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c369bc-3738-4623-a40d-77046a500b02 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:12.744098 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c369bc-3738-4623-a40d-77046a500b02 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:12.744692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c369bc-3738-4623-a40d-77046a500b02 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:12.745443 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c369bc-3738-4623-a40d-77046a500b02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:12.745743 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c369bc-3738-4623-a40d-77046a500b02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:12.747113 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c369bc-3738-4623-a40d-77046a500b02 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:12.755286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 551/1080] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:35:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:35:12.761022 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c369bc-3738-4623-a40d-77046a500b02 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:12.802749 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45c88e-90f5-462b-b750-fd5d3f0287ef None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:12.836699 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7052d1b-078f-4417-90cf-59ecbd9480cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:12.837274 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7052d1b-078f-4417-90cf-59ecbd9480cf None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:12.837545 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7052d1b-078f-4417-90cf-59ecbd9480cf None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:12.837808 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7052d1b-078f-4417-90cf-59ecbd9480cf None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:12.838515 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7052d1b-078f-4417-90cf-59ecbd9480cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:12.838812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7052d1b-078f-4417-90cf-59ecbd9480cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:12.840172 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7052d1b-078f-4417-90cf-59ecbd9480cf None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:12.851382 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 530/1081] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:35:12.854971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c369bc-3738-4623-a40d-77046a500b02 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:12.891234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d7897-40d4-4066-823d-48d2a6d2e91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:12.892133 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d7897-40d4-4066-823d-48d2a6d2e91e None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:12.892405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d7897-40d4-4066-823d-48d2a6d2e91e None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:12.892731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d7897-40d4-4066-823d-48d2a6d2e91e None admin] PATH_INFO: `/v3/users/1044bb831aac48fe84c33327e3646227` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:12.893989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d7897-40d4-4066-823d-48d2a6d2e91e None admin] RBAC: Authorizing `identity:get_user(user_id=1044bb831aac48fe84c33327e3646227)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:12.895467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d7897-40d4-4066-823d-48d2a6d2e91e None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:12.897375 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 552/1082] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:35:12 2022] GET /identity/v3/users/1044bb831aac48fe84c33327e3646227 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:35:12.903035 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7052d1b-078f-4417-90cf-59ecbd9480cf None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:13.005872 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7052d1b-078f-4417-90cf-59ecbd9480cf None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:13.041759 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c222f6-7a7a-4e50-9a23-3bfae2de90ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:13.042647 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c222f6-7a7a-4e50-9a23-3bfae2de90ec None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:13.042919 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c222f6-7a7a-4e50-9a23-3bfae2de90ec None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:13.043184 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c222f6-7a7a-4e50-9a23-3bfae2de90ec None admin] PATH_INFO: `/v3/projects/7725886bec4343b9ade980294d6b0c88` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:13.044365 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c222f6-7a7a-4e50-9a23-3bfae2de90ec None admin] RBAC: Authorizing `identity:get_project(project_id=7725886bec4343b9ade980294d6b0c88)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:13.045843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c222f6-7a7a-4e50-9a23-3bfae2de90ec None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:13.047847 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 531/1083] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:35:12 2022] GET /identity/v3/projects/7725886bec4343b9ade980294d6b0c88 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:13.053735 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d7897-40d4-4066-823d-48d2a6d2e91e None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:13.151533 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d7897-40d4-4066-823d-48d2a6d2e91e None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:13.183011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7355d-a96e-4165-bbfe-b09c6952ea71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:13.183908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7355d-a96e-4165-bbfe-b09c6952ea71 None admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:13.184185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7355d-a96e-4165-bbfe-b09c6952ea71 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:13.184456 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7355d-a96e-4165-bbfe-b09c6952ea71 None admin] PATH_INFO: `/v3/projects/7725886bec4343b9ade980294d6b0c88/users/1044bb831aac48fe84c33327e3646227/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:13.186428 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7355d-a96e-4165-bbfe-b09c6952ea71 None admin] RBAC: Authorizing `identity:create_grant(project_id=7725886bec4343b9ade980294d6b0c88, user_id=1044bb831aac48fe84c33327e3646227, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:13.187867 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7355d-a96e-4165-bbfe-b09c6952ea71 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:13.200562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 553/1084] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:35:13 2022] PUT /identity/v3/projects/7725886bec4343b9ade980294d6b0c88/users/1044bb831aac48fe84c33327e3646227/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:35:13.840273 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe7be8-383c-4c04-b38b-08d5d167432d None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:13.840835 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe7be8-383c-4c04-b38b-08d5d167432d None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:13.840835 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe7be8-383c-4c04-b38b-08d5d167432d None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:13.842099 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 532/1085] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:13.847712 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10eec13-7048-407d-bc2f-ea810add39bd None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:13.848001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10eec13-7048-407d-bc2f-ea810add39bd None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:13.848332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10eec13-7048-407d-bc2f-ea810add39bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:13.866977 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-a10eec13-7048-407d-bc2f-ea810add39bd None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:13.868167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-a10eec13-7048-407d-bc2f-ea810add39bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:13.941335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 554/1086] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:13.964907 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe7be8-383c-4c04-b38b-08d5d167432d None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:14.066577 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe7be8-383c-4c04-b38b-08d5d167432d None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:14.099856 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077230c-3996-4408-8412-27f80750e79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:14.100588 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077230c-3996-4408-8412-27f80750e79b None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:14.100913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077230c-3996-4408-8412-27f80750e79b None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:14.101184 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077230c-3996-4408-8412-27f80750e79b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:14.111891 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077230c-3996-4408-8412-27f80750e79b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:35:14.120283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-8077230c-3996-4408-8412-27f80750e79b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 09:35:14.120283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:14.120283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:14.120283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:14.120283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:14.120283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:14.120283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:14.120283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:14.120283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:14.120283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:14.120283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:14.120283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:14.120283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:35:14.120283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:35:14.120283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:35:14.120283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:35:14.121306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:35:14.121306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:35:14.121306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:35:14.121306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:35:14.121306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:14.121306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:14.121306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:14.121306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:14.121306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:14.121306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:14.121306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:14.121306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:35:14.121306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:14.121306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:14.121306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:14.121306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:14.122370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:14.122370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:14.122370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:14.122370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:14.122370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:35:14.122370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:35:14.122370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:35:14.122370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:35:14.122370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:35:14.122370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:35:14.122370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 09:35:14.122370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:35:14.122370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 533/1087] 10.222.0.26 () {56 vars in 1120 bytes} [Mon May 9 09:35:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:35:14.127574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10eec13-7048-407d-bc2f-ea810add39bd None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:14.229688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10eec13-7048-407d-bc2f-ea810add39bd None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:14.263630 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efc549-bf66-4ff4-baa3-394bc92512b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:14.264283 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efc549-bf66-4ff4-baa3-394bc92512b4 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:14.264563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efc549-bf66-4ff4-baa3-394bc92512b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:14.264868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efc549-bf66-4ff4-baa3-394bc92512b4 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:14.265609 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6efc549-bf66-4ff4-baa3-394bc92512b4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:14.265910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6efc549-bf66-4ff4-baa3-394bc92512b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:14.267274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6efc549-bf66-4ff4-baa3-394bc92512b4 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:14.275479 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 555/1088] 10.222.0.26 () {56 vars in 1115 bytes} [Mon May 9 09:35:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:14.281435 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077230c-3996-4408-8412-27f80750e79b None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:14.382102 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077230c-3996-4408-8412-27f80750e79b None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:14.415720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a73483-2d07-4105-af31-f6145640908f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:14.416506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a73483-2d07-4105-af31-f6145640908f None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:14.416815 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a73483-2d07-4105-af31-f6145640908f None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:14.417088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a73483-2d07-4105-af31-f6145640908f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:14.427248 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a73483-2d07-4105-af31-f6145640908f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:14.428697 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a73483-2d07-4105-af31-f6145640908f None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:14.447078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-a6a73483-2d07-4105-af31-f6145640908f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:35:14.447078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:14.447078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:14.447078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:14.447078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:14.447078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:14.447078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:14.447078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:14.447078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:14.447078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:14.447078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:14.447078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:14.447078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:35:14.447078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:35:14.447078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:35:14.448020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:35:14.448020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:14.448020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:14.448020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:14.448020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:14.448020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:14.448020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:14.448020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:14.448020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:35:14.448020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:14.448020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:14.448020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:14.448020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:14.448020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:14.448020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:14.448020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:14.448020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:14.449219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:35:14.449219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:35:14.449219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:35:14.449219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:35:14.449219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 534/1089] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:35:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:35:14.454148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efc549-bf66-4ff4-baa3-394bc92512b4 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:14.559117 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efc549-bf66-4ff4-baa3-394bc92512b4 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:14.591711 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e2f9b-746e-4b81-b11c-79aa385b4cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:14.592356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e2f9b-746e-4b81-b11c-79aa385b4cc1 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:14.592628 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e2f9b-746e-4b81-b11c-79aa385b4cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:14.592928 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e2f9b-746e-4b81-b11c-79aa385b4cc1 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:14.593635 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e2f9b-746e-4b81-b11c-79aa385b4cc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:14.593931 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e2f9b-746e-4b81-b11c-79aa385b4cc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:14.595288 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e2f9b-746e-4b81-b11c-79aa385b4cc1 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:14.606601 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 556/1090] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:14.612318 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a73483-2d07-4105-af31-f6145640908f None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:14.711692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a73483-2d07-4105-af31-f6145640908f None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:14.745718 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ede6a9-3d0b-44d6-9203-a94281fa75c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:14.746501 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ede6a9-3d0b-44d6-9203-a94281fa75c2 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:14.746782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ede6a9-3d0b-44d6-9203-a94281fa75c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:14.747048 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ede6a9-3d0b-44d6-9203-a94281fa75c2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:14.754175 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ede6a9-3d0b-44d6-9203-a94281fa75c2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:14.755632 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ede6a9-3d0b-44d6-9203-a94281fa75c2 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:14.770016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-b6ede6a9-3d0b-44d6-9203-a94281fa75c2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 09:35:14.770016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:14.770016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:14.770016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:14.770016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:14.770016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:14.770016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:14.770016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:14.770016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:14.770016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:14.770016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:14.770016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:14.770016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:35:14.770016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:35:14.770016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:35:14.770016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:35:14.770985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:14.770985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:14.770985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:35:14.770985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:35:14.770985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:35:14.770985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:35:14.770985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:14.770985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:14.770985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:14.770985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:14.770985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:14.770985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:35:14.770985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:14.770985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:14.770985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:14.770985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:14.770985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:14.770985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:14.772127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:14.772127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:14.772127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:35:14.772127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:35:14.772127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:35:14.772127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:35:14.772127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:35:14.772127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:35:14.772127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 09:35:14.772127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:35:14.772127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 535/1091] 10.222.0.26 () {56 vars in 1093 bytes} [Mon May 9 09:35:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:35:14.777210 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e2f9b-746e-4b81-b11c-79aa385b4cc1 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:14.876030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e2f9b-746e-4b81-b11c-79aa385b4cc1 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:14.909057 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260aab37-aec7-499f-9587-7371f62b509d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:14.909713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260aab37-aec7-499f-9587-7371f62b509d None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:14.909988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260aab37-aec7-499f-9587-7371f62b509d None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:14.910249 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260aab37-aec7-499f-9587-7371f62b509d None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:14.911074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260aab37-aec7-499f-9587-7371f62b509d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:14.911374 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260aab37-aec7-499f-9587-7371f62b509d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:14.912839 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260aab37-aec7-499f-9587-7371f62b509d None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:14.929369 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 557/1092] 10.222.0.26 () {56 vars in 1133 bytes} [Mon May 9 09:35:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:14.935140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ede6a9-3d0b-44d6-9203-a94281fa75c2 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:15.033561 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ede6a9-3d0b-44d6-9203-a94281fa75c2 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:15.066455 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036028f-0e84-4bc6-8282-7013782f02c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:15.067081 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036028f-0e84-4bc6-8282-7013782f02c3 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:15.067351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036028f-0e84-4bc6-8282-7013782f02c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:15.067616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036028f-0e84-4bc6-8282-7013782f02c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:15.136572 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036028f-0e84-4bc6-8282-7013782f02c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:15.138053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036028f-0e84-4bc6-8282-7013782f02c3 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:15.237581 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 536/1093] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:35:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 09:35:15.243384 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260aab37-aec7-499f-9587-7371f62b509d None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:15.339108 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260aab37-aec7-499f-9587-7371f62b509d None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:15.369002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3852aa3-de26-4bf7-ac50-80bfbdf0383e None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:15.369282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3852aa3-de26-4bf7-ac50-80bfbdf0383e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:15.369868 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3852aa3-de26-4bf7-ac50-80bfbdf0383e None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:15.371263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 537/1094] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:15.372206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02e345-1b33-48f7-b29a-1c19de6ba78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:15.373023 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02e345-1b33-48f7-b29a-1c19de6ba78c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:15.373301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02e345-1b33-48f7-b29a-1c19de6ba78c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:15.373567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02e345-1b33-48f7-b29a-1c19de6ba78c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:15.376310 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a5b15-ed14-48f9-9209-0cb9035f4b9d None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:15.376595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a5b15-ed14-48f9-9209-0cb9035f4b9d None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:15.376898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a5b15-ed14-48f9-9209-0cb9035f4b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:15.383560 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b02e345-1b33-48f7-b29a-1c19de6ba78c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:15.385035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b02e345-1b33-48f7-b29a-1c19de6ba78c None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:15.393962 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-949a5b15-ed14-48f9-9209-0cb9035f4b9d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:15.395149 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-949a5b15-ed14-48f9-9209-0cb9035f4b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:15.402541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-3b02e345-1b33-48f7-b29a-1c19de6ba78c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:35:15.402541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:15.402541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:15.402541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:15.402541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:15.402541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:15.402541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:15.402541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:15.402541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:15.402541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:15.402541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:15.402541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:15.402541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:35:15.402541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:35:15.402541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:35:15.403412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:35:15.403412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:15.403412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:15.403412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:15.403412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:15.403412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:15.403412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:15.403412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:15.403412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:35:15.403412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:15.403412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:15.403412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:15.403412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:15.403412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:15.403412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:15.403412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:15.403412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:15.404409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:35:15.404409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:35:15.404409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:35:15.404409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:15.404409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 558/1095] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:35:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:35:15.409371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02e345-1b33-48f7-b29a-1c19de6ba78c None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:15.465922 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 538/1096] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:15.488766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a5b15-ed14-48f9-9209-0cb9035f4b9d None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:15.500269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02e345-1b33-48f7-b29a-1c19de6ba78c None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:15.533328 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a515a-7b2e-4d45-be44-368c7b4e38d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:15.534011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a515a-7b2e-4d45-be44-368c7b4e38d0 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:15.534272 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a515a-7b2e-4d45-be44-368c7b4e38d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:15.534554 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a515a-7b2e-4d45-be44-368c7b4e38d0 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:15.535268 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a515a-7b2e-4d45-be44-368c7b4e38d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:15.535568 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a515a-7b2e-4d45-be44-368c7b4e38d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:15.536969 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a515a-7b2e-4d45-be44-368c7b4e38d0 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:15.551674 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 559/1097] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:35:15.557314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a515a-7b2e-4d45-be44-368c7b4e38d0 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:15.585148 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a5b15-ed14-48f9-9209-0cb9035f4b9d None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:15.619948 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a92d88-3208-4318-9b0b-e4e14b7df266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:15.620689 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a92d88-3208-4318-9b0b-e4e14b7df266 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:15.620960 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a92d88-3208-4318-9b0b-e4e14b7df266 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:15.621230 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a92d88-3208-4318-9b0b-e4e14b7df266 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:15.630866 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a92d88-3208-4318-9b0b-e4e14b7df266 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:35:15.638511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-78a92d88-3208-4318-9b0b-e4e14b7df266 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:35:15.638511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:15.638511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:15.638511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:15.638511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:15.638511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:15.638511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:15.638511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:15.638511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:15.638511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:15.638511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:15.638511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:15.638511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:35:15.638511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:35:15.638511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:35:15.638511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:35:15.639490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:35:15.639490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:35:15.639490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:35:15.639490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:35:15.639490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:15.639490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:15.639490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:15.639490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:15.639490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:15.639490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:15.639490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:15.639490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:35:15.639490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:15.639490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:15.639490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:15.639490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:15.640344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:15.640344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:15.640344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:15.640344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:15.640344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:35:15.640344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:35:15.640344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:35:15.640344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:35:15.640344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:35:15.640344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:35:15.640344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:35:15.640344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:35:15.640344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 539/1098] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:35:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:35:15.645631 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a92d88-3208-4318-9b0b-e4e14b7df266 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:15.654108 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a515a-7b2e-4d45-be44-368c7b4e38d0 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:15.686941 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de58d44-b9c6-4f63-bb27-59c17d7e44ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:15.687726 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de58d44-b9c6-4f63-bb27-59c17d7e44ea None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:15.688005 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de58d44-b9c6-4f63-bb27-59c17d7e44ea None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:15.688271 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de58d44-b9c6-4f63-bb27-59c17d7e44ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:15.694749 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de58d44-b9c6-4f63-bb27-59c17d7e44ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:15.696176 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de58d44-b9c6-4f63-bb27-59c17d7e44ea None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:15.710030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-1de58d44-b9c6-4f63-bb27-59c17d7e44ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:35:15.710030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:15.710030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:15.710030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:15.710030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:15.710030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:15.710030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:15.710030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:15.710030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:15.710030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:15.710030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:15.710030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:15.710030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:35:15.710030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:35:15.710030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:35:15.710988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:35:15.710988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:15.710988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:15.710988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:15.710988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:15.710988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:15.710988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:15.710988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:15.710988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:35:15.710988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:15.710988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:15.710988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:15.710988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:15.710988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:15.710988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:15.710988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:15.710988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:15.712170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:35:15.712170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:35:15.712170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:35:15.712170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:35:15.712170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:35:15.712170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:35:15.712170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:35:15.712170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:15.712170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 560/1099] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:35:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:35:15.717538 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de58d44-b9c6-4f63-bb27-59c17d7e44ea None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:15.738592 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a92d88-3208-4318-9b0b-e4e14b7df266 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:15.769510 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b730a8f-1228-4387-9bc5-88ea4252ef34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:15.770160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b730a8f-1228-4387-9bc5-88ea4252ef34 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:15.770433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b730a8f-1228-4387-9bc5-88ea4252ef34 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:15.770698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b730a8f-1228-4387-9bc5-88ea4252ef34 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:15.771433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b730a8f-1228-4387-9bc5-88ea4252ef34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:15.771730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b730a8f-1228-4387-9bc5-88ea4252ef34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:15.773189 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b730a8f-1228-4387-9bc5-88ea4252ef34 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:15.780841 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 540/1100] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:35:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:15.786685 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b730a8f-1228-4387-9bc5-88ea4252ef34 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:15.813373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de58d44-b9c6-4f63-bb27-59c17d7e44ea None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:15.846458 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d09a4-2d5a-49ec-b19b-e6dc28c0b565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:15.847100 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d09a4-2d5a-49ec-b19b-e6dc28c0b565 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:15.847377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d09a4-2d5a-49ec-b19b-e6dc28c0b565 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:15.847641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d09a4-2d5a-49ec-b19b-e6dc28c0b565 None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:15.848424 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d09a4-2d5a-49ec-b19b-e6dc28c0b565 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:15.848732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d09a4-2d5a-49ec-b19b-e6dc28c0b565 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:15.850241 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d09a4-2d5a-49ec-b19b-e6dc28c0b565 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:15.861508 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 561/1101] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:35:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:35:15.869684 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d09a4-2d5a-49ec-b19b-e6dc28c0b565 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:15.881579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b730a8f-1228-4387-9bc5-88ea4252ef34 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:15.915618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d2503-7b20-4950-b8da-ab7b95551f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:15.916484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d2503-7b20-4950-b8da-ab7b95551f31 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:15.916811 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d2503-7b20-4950-b8da-ab7b95551f31 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:15.917081 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d2503-7b20-4950-b8da-ab7b95551f31 None admin] PATH_INFO: `/v3/users/1044bb831aac48fe84c33327e3646227` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:15.918303 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d2503-7b20-4950-b8da-ab7b95551f31 None admin] RBAC: Authorizing `identity:get_user(user_id=1044bb831aac48fe84c33327e3646227)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:15.919736 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d2503-7b20-4950-b8da-ab7b95551f31 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:15.921407 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 541/1102] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:35:15 2022] GET /identity/v3/users/1044bb831aac48fe84c33327e3646227 => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:35:15.927007 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d2503-7b20-4950-b8da-ab7b95551f31 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:15.961365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d09a4-2d5a-49ec-b19b-e6dc28c0b565 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:15.993654 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ebf28-8eea-4479-adda-5b22c2fe0eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:15.994274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ebf28-8eea-4479-adda-5b22c2fe0eea None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:15.994857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ebf28-8eea-4479-adda-5b22c2fe0eea None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:15.995125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ebf28-8eea-4479-adda-5b22c2fe0eea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:15.995936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ebf28-8eea-4479-adda-5b22c2fe0eea None admin] RBAC: Adding query filter params (role.id=e8b800faebb04b2eb2cebfb7a856d7ad, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=6362303cb918486c8525225914332ad5) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:15.996240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ebf28-8eea-4479-adda-5b22c2fe0eea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:15.997678 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ebf28-8eea-4479-adda-5b22c2fe0eea None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:16.003340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 562/1103] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:35:15 2022] GET /identity/v3/role_assignments?user.id=6362303cb918486c8525225914332ad5&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=e8b800faebb04b2eb2cebfb7a856d7ad => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:35:16.021355 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d2503-7b20-4950-b8da-ab7b95551f31 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:16.058945 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c7c5d-5b67-476b-b0ff-b72e781e3e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:16.059613 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c7c5d-5b67-476b-b0ff-b72e781e3e27 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:16.059888 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c7c5d-5b67-476b-b0ff-b72e781e3e27 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:16.060159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c7c5d-5b67-476b-b0ff-b72e781e3e27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:16.132652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c7c5d-5b67-476b-b0ff-b72e781e3e27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:16.134058 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c7c5d-5b67-476b-b0ff-b72e781e3e27 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:16.233414 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 542/1104] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:35:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 09:35:16.239255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ebf28-8eea-4479-adda-5b22c2fe0eea None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:16.345024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ebf28-8eea-4479-adda-5b22c2fe0eea None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:16.377521 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56b00c-4353-47ef-a646-20984a2fe5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:16.378379 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56b00c-4353-47ef-a646-20984a2fe5f3 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:16.378659 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56b00c-4353-47ef-a646-20984a2fe5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:16.378914 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56b00c-4353-47ef-a646-20984a2fe5f3 None admin] PATH_INFO: `/v3/projects/7725886bec4343b9ade980294d6b0c88` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:16.380145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56b00c-4353-47ef-a646-20984a2fe5f3 None admin] RBAC: Authorizing `identity:get_project(project_id=7725886bec4343b9ade980294d6b0c88)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:16.381629 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56b00c-4353-47ef-a646-20984a2fe5f3 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:16.383410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 563/1105] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:35:16 2022] GET /identity/v3/projects/7725886bec4343b9ade980294d6b0c88 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:16.389518 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c7c5d-5b67-476b-b0ff-b72e781e3e27 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:16.485523 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c7c5d-5b67-476b-b0ff-b72e781e3e27 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:16.519528 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc279ec-51da-4fa4-86ca-f5b1f38d17a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:16.520193 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc279ec-51da-4fa4-86ca-f5b1f38d17a0 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:16.520481 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc279ec-51da-4fa4-86ca-f5b1f38d17a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:16.520776 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc279ec-51da-4fa4-86ca-f5b1f38d17a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:16.521626 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc279ec-51da-4fa4-86ca-f5b1f38d17a0 None admin] RBAC: Adding query filter params (role.id=70ac549382b14b2cbb8b35e5dfc648d4, scope.project.id=7725886bec4343b9ade980294d6b0c88, user.id=1044bb831aac48fe84c33327e3646227) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:16.521934 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc279ec-51da-4fa4-86ca-f5b1f38d17a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:16.523434 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc279ec-51da-4fa4-86ca-f5b1f38d17a0 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:16.529379 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 543/1106] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:35:16 2022] GET /identity/v3/role_assignments?user.id=1044bb831aac48fe84c33327e3646227&scope.project.id=7725886bec4343b9ade980294d6b0c88&role.id=70ac549382b14b2cbb8b35e5dfc648d4 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:18.737655 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcf045-882b-41be-8b7d-66dbfff4dd47 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:18.738544 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcf045-882b-41be-8b7d-66dbfff4dd47 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:18.738544 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcf045-882b-41be-8b7d-66dbfff4dd47 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:18.740732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 564/1107] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:18.747302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113e4b6-a9f4-4434-937a-a268e9f72ad4 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:18.747769 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113e4b6-a9f4-4434-937a-a268e9f72ad4 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:18.747864 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113e4b6-a9f4-4434-937a-a268e9f72ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:18.775493 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-8113e4b6-a9f4-4434-937a-a268e9f72ad4 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:18.777320 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-8113e4b6-a9f4-4434-937a-a268e9f72ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:18.855113 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 544/1108] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:18.890431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edcf045-882b-41be-8b7d-66dbfff4dd47 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:18.998612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edcf045-882b-41be-8b7d-66dbfff4dd47 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:19.031608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235bdb62-8e81-4659-8114-90749a9a9d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:19.032379 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bdb62-8e81-4659-8114-90749a9a9d68 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:19.032650 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bdb62-8e81-4659-8114-90749a9a9d68 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:19.032958 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bdb62-8e81-4659-8114-90749a9a9d68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:19.034164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235bdb62-8e81-4659-8114-90749a9a9d68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:19.035597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235bdb62-8e81-4659-8114-90749a9a9d68 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:19.037240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 565/1109] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:35:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:19.043915 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113e4b6-a9f4-4434-937a-a268e9f72ad4 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:19.170260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113e4b6-a9f4-4434-937a-a268e9f72ad4 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:19.209636 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb3a83-efad-4946-8bee-508f007339ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:19.210375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb3a83-efad-4946-8bee-508f007339ac None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:19.210651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb3a83-efad-4946-8bee-508f007339ac None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:19.210921 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb3a83-efad-4946-8bee-508f007339ac None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:19.211627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb3a83-efad-4946-8bee-508f007339ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:19.213110 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb3a83-efad-4946-8bee-508f007339ac None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:19.234778 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 545/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Mon May 9 09:35:19 2022] POST /identity/v3/users => generated 286 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:35:21.440160 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad8216-bc7b-42c4-9970-bd40c30be1d5 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:21.440794 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad8216-bc7b-42c4-9970-bd40c30be1d5 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:21.440794 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad8216-bc7b-42c4-9970-bd40c30be1d5 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:21.442092 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 566/1111] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:21.448499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bf578-de59-4eec-8f1d-450a1ea31483 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:21.449264 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bf578-de59-4eec-8f1d-450a1ea31483 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:21.449434 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bf578-de59-4eec-8f1d-450a1ea31483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:21.470019 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-f20bf578-de59-4eec-8f1d-450a1ea31483 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:21.471201 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f20bf578-de59-4eec-8f1d-450a1ea31483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:21.538981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 546/1112] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:21.562299 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad8216-bc7b-42c4-9970-bd40c30be1d5 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:21.673042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad8216-bc7b-42c4-9970-bd40c30be1d5 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:21.707385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae3cb8-3d65-4a8d-8a12-85b94f236028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:21.708156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae3cb8-3d65-4a8d-8a12-85b94f236028 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:21.708432 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae3cb8-3d65-4a8d-8a12-85b94f236028 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:21.708710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae3cb8-3d65-4a8d-8a12-85b94f236028 None admin] PATH_INFO: `/v3/roles/0e3ae1b3fdce41c4a42a415efad3b84d` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:21.710353 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae3cb8-3d65-4a8d-8a12-85b94f236028 None admin] RBAC: Authorizing `identity:get_role(role_id=0e3ae1b3fdce41c4a42a415efad3b84d)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:21.711726 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae3cb8-3d65-4a8d-8a12-85b94f236028 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:21.713138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 567/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:35:21 2022] GET /identity/v3/roles/0e3ae1b3fdce41c4a42a415efad3b84d => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:21.719337 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20bf578-de59-4eec-8f1d-450a1ea31483 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:21.822963 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20bf578-de59-4eec-8f1d-450a1ea31483 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:21.857026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1fb2b-ca69-4099-b654-0e6cedb773a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:21.857887 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1fb2b-ca69-4099-b654-0e6cedb773a5 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:21.858163 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1fb2b-ca69-4099-b654-0e6cedb773a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:21.858434 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1fb2b-ca69-4099-b654-0e6cedb773a5 None admin] PATH_INFO: `/v3/users/a8875a0e99a2401ab8e36970acabefaf` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:21.872080 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b1fb2b-ca69-4099-b654-0e6cedb773a5 None admin] RBAC: Authorizing `identity:get_user(user_id=a8875a0e99a2401ab8e36970acabefaf)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:21.873555 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b1fb2b-ca69-4099-b654-0e6cedb773a5 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:21.875182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 547/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:35:21 2022] GET /identity/v3/users/a8875a0e99a2401ab8e36970acabefaf => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:21.883385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae3cb8-3d65-4a8d-8a12-85b94f236028 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:21.993197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae3cb8-3d65-4a8d-8a12-85b94f236028 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:22.028129 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4d02f-cdda-4435-833f-383f8203d0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:22.028780 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4d02f-cdda-4435-833f-383f8203d0cf None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:22.029058 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4d02f-cdda-4435-833f-383f8203d0cf None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:22.029331 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4d02f-cdda-4435-833f-383f8203d0cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:22.102472 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a4d02f-cdda-4435-833f-383f8203d0cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:22.104107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a4d02f-cdda-4435-833f-383f8203d0cf None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:22.206613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 568/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:35:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 09:35:22.214019 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1fb2b-ca69-4099-b654-0e6cedb773a5 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:22.319051 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1fb2b-ca69-4099-b654-0e6cedb773a5 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:22.352976 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af19431-3de1-4545-8143-f3cb9b1ee41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:22.353813 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af19431-3de1-4545-8143-f3cb9b1ee41b None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:22.354084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af19431-3de1-4545-8143-f3cb9b1ee41b None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:22.354349 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af19431-3de1-4545-8143-f3cb9b1ee41b None admin] PATH_INFO: `/v3/projects/7725886bec4343b9ade980294d6b0c88` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:22.355556 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af19431-3de1-4545-8143-f3cb9b1ee41b None admin] RBAC: Authorizing `identity:get_project(project_id=7725886bec4343b9ade980294d6b0c88)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:22.357033 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af19431-3de1-4545-8143-f3cb9b1ee41b None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:22.358731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 548/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:35:22 2022] GET /identity/v3/projects/7725886bec4343b9ade980294d6b0c88 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:22.365804 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4d02f-cdda-4435-833f-383f8203d0cf None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:22.470864 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4d02f-cdda-4435-833f-383f8203d0cf None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:22.506746 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0969ca6-7d73-4b5e-bfc9-a18026f1d271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:22.507380 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0969ca6-7d73-4b5e-bfc9-a18026f1d271 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:22.507655 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0969ca6-7d73-4b5e-bfc9-a18026f1d271 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:22.507925 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0969ca6-7d73-4b5e-bfc9-a18026f1d271 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:22.508788 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0969ca6-7d73-4b5e-bfc9-a18026f1d271 None admin] RBAC: Adding query filter params (role.id=0e3ae1b3fdce41c4a42a415efad3b84d, scope.project.id=7725886bec4343b9ade980294d6b0c88, user.id=a8875a0e99a2401ab8e36970acabefaf) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:22.509090 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0969ca6-7d73-4b5e-bfc9-a18026f1d271 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:22.510532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0969ca6-7d73-4b5e-bfc9-a18026f1d271 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:22.515913 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 569/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:35:22 2022] GET /identity/v3/role_assignments?user.id=a8875a0e99a2401ab8e36970acabefaf&scope.project.id=7725886bec4343b9ade980294d6b0c88&role.id=0e3ae1b3fdce41c4a42a415efad3b84d => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:24.661765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8ed31-c3b3-44c3-b15d-4309ddcbd73f None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:24.662293 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8ed31-c3b3-44c3-b15d-4309ddcbd73f None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:24.662293 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8ed31-c3b3-44c3-b15d-4309ddcbd73f None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:24.663701 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 549/1118] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:24.670116 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55e9d8-df46-40ee-bcfc-a3a31851222b None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:24.670849 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55e9d8-df46-40ee-bcfc-a3a31851222b None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:24.670992 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55e9d8-df46-40ee-bcfc-a3a31851222b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:24.692058 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-7b55e9d8-df46-40ee-bcfc-a3a31851222b None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:24.693289 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-7b55e9d8-df46-40ee-bcfc-a3a31851222b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:24.760351 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 570/1119] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:24.787619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8ed31-c3b3-44c3-b15d-4309ddcbd73f None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:24.892254 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8ed31-c3b3-44c3-b15d-4309ddcbd73f None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:24.930932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217b942-0af7-495f-84ff-34709cc279cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:24.931780 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217b942-0af7-495f-84ff-34709cc279cf None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:24.932055 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217b942-0af7-495f-84ff-34709cc279cf None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:24.932326 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217b942-0af7-495f-84ff-34709cc279cf None admin] PATH_INFO: `/v3/roles/0e3ae1b3fdce41c4a42a415efad3b84d` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:24.934098 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217b942-0af7-495f-84ff-34709cc279cf None admin] RBAC: Authorizing `identity:get_role(role_id=0e3ae1b3fdce41c4a42a415efad3b84d)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:24.935556 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217b942-0af7-495f-84ff-34709cc279cf None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:24.941046 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 550/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:35:24 2022] GET /identity/v3/roles/0e3ae1b3fdce41c4a42a415efad3b84d => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:24.951840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55e9d8-df46-40ee-bcfc-a3a31851222b None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:25.062335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55e9d8-df46-40ee-bcfc-a3a31851222b None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:25.095494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bfd84-f4e5-4861-a371-fdbacc74cf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:25.096420 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bfd84-f4e5-4861-a371-fdbacc74cf2c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:25.096709 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bfd84-f4e5-4861-a371-fdbacc74cf2c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:25.096996 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bfd84-f4e5-4861-a371-fdbacc74cf2c None admin] PATH_INFO: `/v3/users/a8875a0e99a2401ab8e36970acabefaf` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:25.098224 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bfd84-f4e5-4861-a371-fdbacc74cf2c None admin] RBAC: Authorizing `identity:get_user(user_id=a8875a0e99a2401ab8e36970acabefaf)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:25.099652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bfd84-f4e5-4861-a371-fdbacc74cf2c None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:25.101371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 571/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:35:24 2022] GET /identity/v3/users/a8875a0e99a2401ab8e36970acabefaf => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:25.107884 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217b942-0af7-495f-84ff-34709cc279cf None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:25.213990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217b942-0af7-495f-84ff-34709cc279cf None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:25.247894 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edac41-e2f8-452e-9596-27f01080bd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:25.248728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edac41-e2f8-452e-9596-27f01080bd50 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:25.249033 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edac41-e2f8-452e-9596-27f01080bd50 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:25.249285 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edac41-e2f8-452e-9596-27f01080bd50 None admin] PATH_INFO: `/v3/projects/7725886bec4343b9ade980294d6b0c88` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:25.250577 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edac41-e2f8-452e-9596-27f01080bd50 None admin] RBAC: Authorizing `identity:get_project(project_id=7725886bec4343b9ade980294d6b0c88)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:25.251997 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edac41-e2f8-452e-9596-27f01080bd50 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:25.253960 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 551/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:35:25 2022] GET /identity/v3/projects/7725886bec4343b9ade980294d6b0c88 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:25.260106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bfd84-f4e5-4861-a371-fdbacc74cf2c None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:25.362370 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bfd84-f4e5-4861-a371-fdbacc74cf2c None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:25.396677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8d17d-4992-41c2-95fb-2eefcb607fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:25.397624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8d17d-4992-41c2-95fb-2eefcb607fbf None admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:25.397898 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8d17d-4992-41c2-95fb-2eefcb607fbf None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:25.398170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8d17d-4992-41c2-95fb-2eefcb607fbf None admin] PATH_INFO: `/v3/projects/7725886bec4343b9ade980294d6b0c88/users/a8875a0e99a2401ab8e36970acabefaf/roles/0e3ae1b3fdce41c4a42a415efad3b84d` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:25.400856 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e8d17d-4992-41c2-95fb-2eefcb607fbf None admin] RBAC: Authorizing `identity:create_grant(project_id=7725886bec4343b9ade980294d6b0c88, user_id=a8875a0e99a2401ab8e36970acabefaf, role_id=0e3ae1b3fdce41c4a42a415efad3b84d)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:25.402314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e8d17d-4992-41c2-95fb-2eefcb607fbf None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:25.420366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 572/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:35:25 2022] PUT /identity/v3/projects/7725886bec4343b9ade980294d6b0c88/users/a8875a0e99a2401ab8e36970acabefaf/roles/0e3ae1b3fdce41c4a42a415efad3b84d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:35:27.570171 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb4836-d4ca-4e1c-bba0-80eaa02cb0ea None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:27.570620 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb4836-d4ca-4e1c-bba0-80eaa02cb0ea None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:27.570712 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb4836-d4ca-4e1c-bba0-80eaa02cb0ea None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:27.572068 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 552/1124] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:27.578290 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b77c12-eadc-4666-bd3c-8552af1c2427 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:27.579006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b77c12-eadc-4666-bd3c-8552af1c2427 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:27.579110 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b77c12-eadc-4666-bd3c-8552af1c2427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:27.598279 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-26b77c12-eadc-4666-bd3c-8552af1c2427 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:27.599457 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-26b77c12-eadc-4666-bd3c-8552af1c2427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:27.670992 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 573/1125] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:27.694073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb4836-d4ca-4e1c-bba0-80eaa02cb0ea None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:27.795627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb4836-d4ca-4e1c-bba0-80eaa02cb0ea None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:27.829166 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd86f31-9238-4bb7-9bd2-4db124c33b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:27.829936 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd86f31-9238-4bb7-9bd2-4db124c33b39 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:27.830207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd86f31-9238-4bb7-9bd2-4db124c33b39 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:27.830473 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd86f31-9238-4bb7-9bd2-4db124c33b39 None admin] PATH_INFO: `/v3/roles/0e3ae1b3fdce41c4a42a415efad3b84d` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:27.832049 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd86f31-9238-4bb7-9bd2-4db124c33b39 None admin] RBAC: Authorizing `identity:get_role(role_id=0e3ae1b3fdce41c4a42a415efad3b84d)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:27.833435 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd86f31-9238-4bb7-9bd2-4db124c33b39 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:27.834819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 553/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:35:27 2022] GET /identity/v3/roles/0e3ae1b3fdce41c4a42a415efad3b84d => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:27.840965 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b77c12-eadc-4666-bd3c-8552af1c2427 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:27.932187 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b77c12-eadc-4666-bd3c-8552af1c2427 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:27.968722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620dfd7-bdbe-4cb1-b1eb-4bcb010d7edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:27.969649 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620dfd7-bdbe-4cb1-b1eb-4bcb010d7edc None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:27.969947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620dfd7-bdbe-4cb1-b1eb-4bcb010d7edc None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:27.970210 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620dfd7-bdbe-4cb1-b1eb-4bcb010d7edc None admin] PATH_INFO: `/v3/users/a8875a0e99a2401ab8e36970acabefaf` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:27.971502 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d620dfd7-bdbe-4cb1-b1eb-4bcb010d7edc None admin] RBAC: Authorizing `identity:get_user(user_id=a8875a0e99a2401ab8e36970acabefaf)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:27.972955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d620dfd7-bdbe-4cb1-b1eb-4bcb010d7edc None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:27.974566 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 574/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:35:27 2022] GET /identity/v3/users/a8875a0e99a2401ab8e36970acabefaf => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:27.980689 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd86f31-9238-4bb7-9bd2-4db124c33b39 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:28.082713 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd86f31-9238-4bb7-9bd2-4db124c33b39 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:28.119853 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28e1bc-fbe7-4822-ba6c-88749d307ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:28.120476 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28e1bc-fbe7-4822-ba6c-88749d307ad6 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:28.120779 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28e1bc-fbe7-4822-ba6c-88749d307ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:28.121042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28e1bc-fbe7-4822-ba6c-88749d307ad6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:28.190269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28e1bc-fbe7-4822-ba6c-88749d307ad6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:28.191690 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28e1bc-fbe7-4822-ba6c-88749d307ad6 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:28.286990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 554/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:35:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 09:35:28.295680 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620dfd7-bdbe-4cb1-b1eb-4bcb010d7edc None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:28.392078 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620dfd7-bdbe-4cb1-b1eb-4bcb010d7edc None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:28.425278 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa24a99-afc4-4b40-9adf-e61e53856764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:28.426114 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa24a99-afc4-4b40-9adf-e61e53856764 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:28.426383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa24a99-afc4-4b40-9adf-e61e53856764 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:28.426650 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa24a99-afc4-4b40-9adf-e61e53856764 None admin] PATH_INFO: `/v3/projects/7725886bec4343b9ade980294d6b0c88` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:28.427933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa24a99-afc4-4b40-9adf-e61e53856764 None admin] RBAC: Authorizing `identity:get_project(project_id=7725886bec4343b9ade980294d6b0c88)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:28.429450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa24a99-afc4-4b40-9adf-e61e53856764 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:28.431090 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 575/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:35:28 2022] GET /identity/v3/projects/7725886bec4343b9ade980294d6b0c88 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:35:28.438091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28e1bc-fbe7-4822-ba6c-88749d307ad6 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:28.549687 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28e1bc-fbe7-4822-ba6c-88749d307ad6 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:28.588976 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d772094-172e-4f3d-94fe-3901f22b93cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:28.589601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d772094-172e-4f3d-94fe-3901f22b93cb None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:28.589877 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d772094-172e-4f3d-94fe-3901f22b93cb None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:28.590141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d772094-172e-4f3d-94fe-3901f22b93cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:28.590962 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d772094-172e-4f3d-94fe-3901f22b93cb None admin] RBAC: Adding query filter params (role.id=0e3ae1b3fdce41c4a42a415efad3b84d, scope.project.id=7725886bec4343b9ade980294d6b0c88, user.id=a8875a0e99a2401ab8e36970acabefaf) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:28.591265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d772094-172e-4f3d-94fe-3901f22b93cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:28.592705 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d772094-172e-4f3d-94fe-3901f22b93cb None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:28.598598 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 555/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:35:28 2022] GET /identity/v3/role_assignments?user.id=a8875a0e99a2401ab8e36970acabefaf&scope.project.id=7725886bec4343b9ade980294d6b0c88&role.id=0e3ae1b3fdce41c4a42a415efad3b84d => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:30.792541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26930498-77b6-486d-b3a9-44c2e088ea5e None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:30.793519 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26930498-77b6-486d-b3a9-44c2e088ea5e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:30.793519 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26930498-77b6-486d-b3a9-44c2e088ea5e None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:30.795630 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 576/1131] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:30.801947 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a00896-ff41-446d-8f0d-139a3a7ccf4c None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:30.802643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a00896-ff41-446d-8f0d-139a3a7ccf4c None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:30.802827 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a00896-ff41-446d-8f0d-139a3a7ccf4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:30.821380 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-d5a00896-ff41-446d-8f0d-139a3a7ccf4c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:30.822487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-d5a00896-ff41-446d-8f0d-139a3a7ccf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:30.888963 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 556/1132] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:30.912381 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26930498-77b6-486d-b3a9-44c2e088ea5e None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:31.035348 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26930498-77b6-486d-b3a9-44c2e088ea5e None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:31.075555 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7890fa8-979f-4aed-8e2a-7cfc8b7ce466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:31.076619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7890fa8-979f-4aed-8e2a-7cfc8b7ce466 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:31.077101 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7890fa8-979f-4aed-8e2a-7cfc8b7ce466 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:31.077523 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7890fa8-979f-4aed-8e2a-7cfc8b7ce466 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:31.079648 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7890fa8-979f-4aed-8e2a-7cfc8b7ce466 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:31.082019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7890fa8-979f-4aed-8e2a-7cfc8b7ce466 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:31.084565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 577/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:35:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:31.090349 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a00896-ff41-446d-8f0d-139a3a7ccf4c None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:31.190521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a00896-ff41-446d-8f0d-139a3a7ccf4c None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:31.224852 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9438ec7-8316-47b2-b7b6-58fbafc3307d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:31.225504 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9438ec7-8316-47b2-b7b6-58fbafc3307d None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:31.225774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9438ec7-8316-47b2-b7b6-58fbafc3307d None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:31.226035 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9438ec7-8316-47b2-b7b6-58fbafc3307d None admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:31.226732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9438ec7-8316-47b2-b7b6-58fbafc3307d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:31.228151 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9438ec7-8316-47b2-b7b6-58fbafc3307d None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:31.260443 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 557/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Mon May 9 09:35:31 2022] POST /identity/v3/projects => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:35:33.510023 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74996ccc-399e-4938-b601-90ea879dc985 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:33.510619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74996ccc-399e-4938-b601-90ea879dc985 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:33.510619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74996ccc-399e-4938-b601-90ea879dc985 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:33.511924 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 578/1135] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:33.518437 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8f2d8-ce85-4ede-be9d-b33b0966a9c9 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:33.519198 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8f2d8-ce85-4ede-be9d-b33b0966a9c9 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:33.519338 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8f2d8-ce85-4ede-be9d-b33b0966a9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:33.539435 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-91c8f2d8-ce85-4ede-be9d-b33b0966a9c9 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:33.540671 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-91c8f2d8-ce85-4ede-be9d-b33b0966a9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:33.613184 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 558/1136] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:33.638197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74996ccc-399e-4938-b601-90ea879dc985 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:33.747764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74996ccc-399e-4938-b601-90ea879dc985 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:33.781472 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8412d-4946-4f00-85af-a324f528b223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:33.782180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8412d-4946-4f00-85af-a324f528b223 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:33.782460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8412d-4946-4f00-85af-a324f528b223 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:33.782725 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8412d-4946-4f00-85af-a324f528b223 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:33.784002 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8412d-4946-4f00-85af-a324f528b223 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:33.785465 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8412d-4946-4f00-85af-a324f528b223 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:33.787100 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 579/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:35:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:33.793214 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c8f2d8-ce85-4ede-be9d-b33b0966a9c9 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:33.891916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c8f2d8-ce85-4ede-be9d-b33b0966a9c9 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:33.928353 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c6b62-a9eb-41be-9ef3-00ceedea63e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:33.929037 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c6b62-a9eb-41be-9ef3-00ceedea63e5 None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:33.929314 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c6b62-a9eb-41be-9ef3-00ceedea63e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:33.929579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c6b62-a9eb-41be-9ef3-00ceedea63e5 None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:33.930294 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c6b62-a9eb-41be-9ef3-00ceedea63e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:33.931723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c6b62-a9eb-41be-9ef3-00ceedea63e5 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:33.953611 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 559/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Mon May 9 09:35:33 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:35:36.203528 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31f743-8dd4-486d-b956-ca84a846416a None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:36.204448 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31f743-8dd4-486d-b956-ca84a846416a None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:36.204448 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31f743-8dd4-486d-b956-ca84a846416a None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:36.206575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 580/1139] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:36.215147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3456f-5a95-4761-bbfe-92bbf5a50dd5 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:36.215881 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3456f-5a95-4761-bbfe-92bbf5a50dd5 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:36.216030 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3456f-5a95-4761-bbfe-92bbf5a50dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:36.236958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-8ba3456f-5a95-4761-bbfe-92bbf5a50dd5 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:36.238142 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-8ba3456f-5a95-4761-bbfe-92bbf5a50dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:36.304227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 560/1140] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:36.327220 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a31f743-8dd4-486d-b956-ca84a846416a None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:36.431868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a31f743-8dd4-486d-b956-ca84a846416a None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:36.465614 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb2f13-f846-44e5-8945-faff6f27422c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:36.466317 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb2f13-f846-44e5-8945-faff6f27422c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:36.466593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb2f13-f846-44e5-8945-faff6f27422c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:36.466858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb2f13-f846-44e5-8945-faff6f27422c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:36.476533 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb2f13-f846-44e5-8945-faff6f27422c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:35:36.484035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-12eb2f13-f846-44e5-8945-faff6f27422c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:35:36.484035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:36.484035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:36.484035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:36.484035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:36.484035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:36.484035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:36.484035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:36.484035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:36.484035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:36.484035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:36.484035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:36.484035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:35:36.484035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:35:36.484035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:35:36.484035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:35:36.485156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:35:36.485156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:35:36.485156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:35:36.485156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:35:36.485156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:36.485156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:36.485156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:36.485156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:36.485156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:36.485156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:36.485156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:36.485156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:35:36.485156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:36.485156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:36.485156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:36.485156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:36.486153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:36.486153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:36.486153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:36.486153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:36.486153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:35:36.486153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:35:36.486153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:35:36.486153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:35:36.486153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:35:36.486153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:35:36.486153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:35:36.486153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:36.486153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 581/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:35:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:35:36.491888 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3456f-5a95-4761-bbfe-92bbf5a50dd5 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:36.614864 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3456f-5a95-4761-bbfe-92bbf5a50dd5 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:36.650236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a9f83-70a1-4679-9c49-08b589cab733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:36.650897 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a9f83-70a1-4679-9c49-08b589cab733 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:36.651169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a9f83-70a1-4679-9c49-08b589cab733 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:36.651439 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a9f83-70a1-4679-9c49-08b589cab733 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:36.652187 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a9f83-70a1-4679-9c49-08b589cab733 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:36.652485 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a9f83-70a1-4679-9c49-08b589cab733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:36.653915 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a9f83-70a1-4679-9c49-08b589cab733 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:36.661860 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 561/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:35:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:36.668473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb2f13-f846-44e5-8945-faff6f27422c None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:36.769892 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb2f13-f846-44e5-8945-faff6f27422c None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:36.802061 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2cf04-0c9d-4018-8f9b-654d48e14d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:36.802967 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2cf04-0c9d-4018-8f9b-654d48e14d1b None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:36.803241 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2cf04-0c9d-4018-8f9b-654d48e14d1b None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:36.803503 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2cf04-0c9d-4018-8f9b-654d48e14d1b None admin] PATH_INFO: `/v3/users/92769800179d41b280e626c794016d37` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:36.816017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be2cf04-0c9d-4018-8f9b-654d48e14d1b None admin] RBAC: Authorizing `identity:get_user(user_id=92769800179d41b280e626c794016d37)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:36.817471 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be2cf04-0c9d-4018-8f9b-654d48e14d1b None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:36.819156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 582/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:35:36 2022] GET /identity/v3/users/92769800179d41b280e626c794016d37 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:36.828381 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a9f83-70a1-4679-9c49-08b589cab733 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:36.934736 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a9f83-70a1-4679-9c49-08b589cab733 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:36.967708 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93a111-c1b1-4994-a538-1a45b8f800f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:36.968328 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93a111-c1b1-4994-a538-1a45b8f800f5 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:36.968599 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93a111-c1b1-4994-a538-1a45b8f800f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:36.968893 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93a111-c1b1-4994-a538-1a45b8f800f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:37.035713 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93a111-c1b1-4994-a538-1a45b8f800f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:37.037250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93a111-c1b1-4994-a538-1a45b8f800f5 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:37.134376 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 562/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:35:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 09:35:37.140571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2cf04-0c9d-4018-8f9b-654d48e14d1b None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:37.239527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2cf04-0c9d-4018-8f9b-654d48e14d1b None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:37.274571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a044d8-3fe6-4834-8f9b-20db72854280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:37.275405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a044d8-3fe6-4834-8f9b-20db72854280 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:37.275677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a044d8-3fe6-4834-8f9b-20db72854280 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:37.275943 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a044d8-3fe6-4834-8f9b-20db72854280 None admin] PATH_INFO: `/v3/projects/897e7fd02f6b45a2a62a05ad63e09265` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:37.277170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a044d8-3fe6-4834-8f9b-20db72854280 None admin] RBAC: Authorizing `identity:get_project(project_id=897e7fd02f6b45a2a62a05ad63e09265)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:37.278597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a044d8-3fe6-4834-8f9b-20db72854280 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:37.280245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 583/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:35:37 2022] GET /identity/v3/projects/897e7fd02f6b45a2a62a05ad63e09265 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:37.289725 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93a111-c1b1-4994-a538-1a45b8f800f5 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:37.401252 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93a111-c1b1-4994-a538-1a45b8f800f5 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:37.435839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4105f00-e573-484d-b9c1-f89a4d4528f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:37.436479 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4105f00-e573-484d-b9c1-f89a4d4528f3 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:37.436784 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4105f00-e573-484d-b9c1-f89a4d4528f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:37.437053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4105f00-e573-484d-b9c1-f89a4d4528f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:37.437876 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4105f00-e573-484d-b9c1-f89a4d4528f3 None admin] RBAC: Adding query filter params (role.id=70ac549382b14b2cbb8b35e5dfc648d4, scope.project.id=897e7fd02f6b45a2a62a05ad63e09265, user.id=92769800179d41b280e626c794016d37) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:37.438183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4105f00-e573-484d-b9c1-f89a4d4528f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:37.439609 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4105f00-e573-484d-b9c1-f89a4d4528f3 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:37.444788 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 563/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:35:37 2022] GET /identity/v3/role_assignments?user.id=92769800179d41b280e626c794016d37&scope.project.id=897e7fd02f6b45a2a62a05ad63e09265&role.id=70ac549382b14b2cbb8b35e5dfc648d4 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:39.611043 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d47040-e075-4440-817e-3ae0fcffabff None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:39.611829 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d47040-e075-4440-817e-3ae0fcffabff None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:39.611829 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d47040-e075-4440-817e-3ae0fcffabff None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:39.614057 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 584/1147] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:39.621218 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a371a8-4581-4d9c-8c12-2408b093582e None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:39.621636 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a371a8-4581-4d9c-8c12-2408b093582e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:39.621783 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a371a8-4581-4d9c-8c12-2408b093582e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:39.648619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-61a371a8-4581-4d9c-8c12-2408b093582e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:39.649831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-61a371a8-4581-4d9c-8c12-2408b093582e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:39.722782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 564/1148] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:39.745983 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d47040-e075-4440-817e-3ae0fcffabff None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:39.847823 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d47040-e075-4440-817e-3ae0fcffabff None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:39.882379 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c9f05-58ac-4aa0-8459-5bbd3aac7adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:39.883165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c9f05-58ac-4aa0-8459-5bbd3aac7adc None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:39.883438 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c9f05-58ac-4aa0-8459-5bbd3aac7adc None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:39.883700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c9f05-58ac-4aa0-8459-5bbd3aac7adc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:39.894219 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c9f05-58ac-4aa0-8459-5bbd3aac7adc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:35:39.901771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-751c9f05-58ac-4aa0-8459-5bbd3aac7adc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:35:39.901771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:39.901771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:39.901771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:39.901771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:39.901771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:39.901771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:39.901771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:39.901771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:39.901771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:39.901771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:39.901771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:39.901771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:35:39.901771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:35:39.901771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:35:39.901771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:35:39.902848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:35:39.902848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:35:39.902848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:35:39.902848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:35:39.902848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:39.902848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:39.902848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:39.902848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:39.902848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:39.902848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:39.902848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:39.902848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:35:39.902848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:39.902848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:39.902848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:39.902848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:39.903729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:39.903729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:39.903729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:39.903729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:39.903729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:35:39.903729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:35:39.903729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:35:39.903729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:35:39.903729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:35:39.903729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:35:39.903729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:35:39.903729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:39.903729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 585/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:35:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:35:39.910960 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a371a8-4581-4d9c-8c12-2408b093582e None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:40.010585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a371a8-4581-4d9c-8c12-2408b093582e None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:40.043875 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dff5e9-1974-4943-ae24-75198604035e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:40.044523 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dff5e9-1974-4943-ae24-75198604035e None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:40.044826 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dff5e9-1974-4943-ae24-75198604035e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:40.045091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dff5e9-1974-4943-ae24-75198604035e None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:40.045838 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dff5e9-1974-4943-ae24-75198604035e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:40.046142 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dff5e9-1974-4943-ae24-75198604035e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:40.047500 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dff5e9-1974-4943-ae24-75198604035e None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:40.055383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 565/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:35:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:40.062487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c9f05-58ac-4aa0-8459-5bbd3aac7adc None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:40.157934 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c9f05-58ac-4aa0-8459-5bbd3aac7adc None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:40.199950 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b3394-1411-4b15-9e7e-d58ac31bb6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:40.200844 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b3394-1411-4b15-9e7e-d58ac31bb6f3 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:40.201125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b3394-1411-4b15-9e7e-d58ac31bb6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:40.201396 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b3394-1411-4b15-9e7e-d58ac31bb6f3 None admin] PATH_INFO: `/v3/users/92769800179d41b280e626c794016d37` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:40.202651 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b3394-1411-4b15-9e7e-d58ac31bb6f3 None admin] RBAC: Authorizing `identity:get_user(user_id=92769800179d41b280e626c794016d37)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:40.204090 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b3394-1411-4b15-9e7e-d58ac31bb6f3 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:40.205741 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 586/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:35:40 2022] GET /identity/v3/users/92769800179d41b280e626c794016d37 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:40.211539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dff5e9-1974-4943-ae24-75198604035e None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:40.320177 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dff5e9-1974-4943-ae24-75198604035e None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:40.358838 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9e255-5ec0-444e-90ff-c87f42201463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:40.359696 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9e255-5ec0-444e-90ff-c87f42201463 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:40.359973 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9e255-5ec0-444e-90ff-c87f42201463 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:40.360242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9e255-5ec0-444e-90ff-c87f42201463 None admin] PATH_INFO: `/v3/projects/897e7fd02f6b45a2a62a05ad63e09265` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:40.361558 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d9e255-5ec0-444e-90ff-c87f42201463 None admin] RBAC: Authorizing `identity:get_project(project_id=897e7fd02f6b45a2a62a05ad63e09265)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:40.362991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d9e255-5ec0-444e-90ff-c87f42201463 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:40.364704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 566/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:35:40 2022] GET /identity/v3/projects/897e7fd02f6b45a2a62a05ad63e09265 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:40.372008 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b3394-1411-4b15-9e7e-d58ac31bb6f3 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:40.476388 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b3394-1411-4b15-9e7e-d58ac31bb6f3 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:40.509377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85f009-026d-4157-9dc8-1b9257a6bd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:40.510240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85f009-026d-4157-9dc8-1b9257a6bd23 None admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:40.510512 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85f009-026d-4157-9dc8-1b9257a6bd23 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:40.510776 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85f009-026d-4157-9dc8-1b9257a6bd23 None admin] PATH_INFO: `/v3/projects/897e7fd02f6b45a2a62a05ad63e09265/users/92769800179d41b280e626c794016d37/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:40.512969 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85f009-026d-4157-9dc8-1b9257a6bd23 None admin] RBAC: Authorizing `identity:create_grant(project_id=897e7fd02f6b45a2a62a05ad63e09265, user_id=92769800179d41b280e626c794016d37, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:40.514378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85f009-026d-4157-9dc8-1b9257a6bd23 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:40.530839 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 587/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:35:40 2022] PUT /identity/v3/projects/897e7fd02f6b45a2a62a05ad63e09265/users/92769800179d41b280e626c794016d37/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:35:42.698161 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c8980-7811-4b88-8f5f-6f5c229d7295 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:42.698714 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c8980-7811-4b88-8f5f-6f5c229d7295 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:42.698714 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c8980-7811-4b88-8f5f-6f5c229d7295 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:42.700088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 567/1154] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:42.706484 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be31cfa-96ca-452a-a294-d2cb3c91082f None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:42.707271 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be31cfa-96ca-452a-a294-d2cb3c91082f None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:42.707426 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be31cfa-96ca-452a-a294-d2cb3c91082f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:42.726769 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-0be31cfa-96ca-452a-a294-d2cb3c91082f None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:42.727956 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-0be31cfa-96ca-452a-a294-d2cb3c91082f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:42.796486 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 588/1155] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:42.829076 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c8980-7811-4b88-8f5f-6f5c229d7295 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:42.932428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c8980-7811-4b88-8f5f-6f5c229d7295 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:42.966378 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa9a52-7b7f-481d-bd5c-01e1560f95e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:42.967163 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa9a52-7b7f-481d-bd5c-01e1560f95e1 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:42.967436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa9a52-7b7f-481d-bd5c-01e1560f95e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:42.967703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa9a52-7b7f-481d-bd5c-01e1560f95e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:42.981225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa9a52-7b7f-481d-bd5c-01e1560f95e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:35:42.988754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-27fa9a52-7b7f-481d-bd5c-01e1560f95e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:35:42.988754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:42.988754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:42.988754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:42.988754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:42.988754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:42.988754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:42.988754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:42.988754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:42.988754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:42.988754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:42.988754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:42.988754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:35:42.988754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:35:42.988754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:35:42.988754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:35:42.989871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:35:42.989871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:35:42.989871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:35:42.989871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:35:42.989871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:42.989871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:42.989871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:42.989871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:42.989871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:42.989871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:42.989871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:42.989871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:35:42.989871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:42.989871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:42.989871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:42.989871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:42.990981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:42.990981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:42.990981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:42.990981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:42.990981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:35:42.990981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:35:42.990981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:35:42.990981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:35:42.990981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:35:42.990981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:35:42.990981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:35:42.990981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:35:42.990981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 568/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:35:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:35:42.996329 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be31cfa-96ca-452a-a294-d2cb3c91082f None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:43.099736 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be31cfa-96ca-452a-a294-d2cb3c91082f None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:43.133257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94aaba5-73ca-43c2-ac29-6843e11e0326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:43.133907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94aaba5-73ca-43c2-ac29-6843e11e0326 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:43.134187 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94aaba5-73ca-43c2-ac29-6843e11e0326 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:43.134460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94aaba5-73ca-43c2-ac29-6843e11e0326 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:43.135191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94aaba5-73ca-43c2-ac29-6843e11e0326 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:43.135493 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94aaba5-73ca-43c2-ac29-6843e11e0326 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:43.136906 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94aaba5-73ca-43c2-ac29-6843e11e0326 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:43.144726 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 589/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:35:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:43.151100 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa9a52-7b7f-481d-bd5c-01e1560f95e1 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:43.253775 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa9a52-7b7f-481d-bd5c-01e1560f95e1 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:43.288468 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f44371-345a-4c70-83fe-c38139a2102a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:43.289357 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f44371-345a-4c70-83fe-c38139a2102a None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:43.289633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f44371-345a-4c70-83fe-c38139a2102a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:43.289903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f44371-345a-4c70-83fe-c38139a2102a None admin] PATH_INFO: `/v3/users/92769800179d41b280e626c794016d37` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:43.291130 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f44371-345a-4c70-83fe-c38139a2102a None admin] RBAC: Authorizing `identity:get_user(user_id=92769800179d41b280e626c794016d37)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:43.292563 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f44371-345a-4c70-83fe-c38139a2102a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:43.294192 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 569/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:35:43 2022] GET /identity/v3/users/92769800179d41b280e626c794016d37 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:43.303132 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94aaba5-73ca-43c2-ac29-6843e11e0326 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:43.404719 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94aaba5-73ca-43c2-ac29-6843e11e0326 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:43.437234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a8d364-888c-4892-94de-34307917066b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:43.437863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8d364-888c-4892-94de-34307917066b None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:43.438141 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8d364-888c-4892-94de-34307917066b None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:43.438410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8d364-888c-4892-94de-34307917066b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:43.503148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a8d364-888c-4892-94de-34307917066b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:43.504579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a8d364-888c-4892-94de-34307917066b None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:43.616387 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 590/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:35:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 09:35:43.622844 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f44371-345a-4c70-83fe-c38139a2102a None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:43.722790 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f44371-345a-4c70-83fe-c38139a2102a None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:43.757242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5078c8-2a89-4042-8a64-3063a27f97f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:43.758090 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5078c8-2a89-4042-8a64-3063a27f97f7 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:43.758363 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5078c8-2a89-4042-8a64-3063a27f97f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:43.758630 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5078c8-2a89-4042-8a64-3063a27f97f7 None admin] PATH_INFO: `/v3/projects/897e7fd02f6b45a2a62a05ad63e09265` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:43.759845 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5078c8-2a89-4042-8a64-3063a27f97f7 None admin] RBAC: Authorizing `identity:get_project(project_id=897e7fd02f6b45a2a62a05ad63e09265)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:43.761311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5078c8-2a89-4042-8a64-3063a27f97f7 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:43.762999 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 570/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:35:43 2022] GET /identity/v3/projects/897e7fd02f6b45a2a62a05ad63e09265 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:43.770046 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a8d364-888c-4892-94de-34307917066b None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:43.879772 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a8d364-888c-4892-94de-34307917066b None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:43.913302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497bc478-25e3-42d3-8857-68fd4d25558c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:43.913940 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bc478-25e3-42d3-8857-68fd4d25558c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:43.914212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bc478-25e3-42d3-8857-68fd4d25558c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:43.914483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bc478-25e3-42d3-8857-68fd4d25558c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:43.915563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497bc478-25e3-42d3-8857-68fd4d25558c None admin] RBAC: Adding query filter params (role.id=70ac549382b14b2cbb8b35e5dfc648d4, scope.project.id=897e7fd02f6b45a2a62a05ad63e09265, user.id=92769800179d41b280e626c794016d37) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:43.915875 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497bc478-25e3-42d3-8857-68fd4d25558c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:43.917332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497bc478-25e3-42d3-8857-68fd4d25558c None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:43.923107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 591/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:35:43 2022] GET /identity/v3/role_assignments?user.id=92769800179d41b280e626c794016d37&scope.project.id=897e7fd02f6b45a2a62a05ad63e09265&role.id=70ac549382b14b2cbb8b35e5dfc648d4 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:46.155140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca40ff-5767-4454-a08a-e5221f6d9fce None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:46.155739 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca40ff-5767-4454-a08a-e5221f6d9fce None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:46.155739 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca40ff-5767-4454-a08a-e5221f6d9fce None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:46.157094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 571/1162] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:46.164056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c4195-c0a7-4d61-9f14-a73ecbd69858 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:46.164788 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c4195-c0a7-4d61-9f14-a73ecbd69858 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:46.164983 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c4195-c0a7-4d61-9f14-a73ecbd69858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:46.187441 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-710c4195-c0a7-4d61-9f14-a73ecbd69858 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:46.188648 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-710c4195-c0a7-4d61-9f14-a73ecbd69858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:46.253448 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 592/1163] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:46.283296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca40ff-5767-4454-a08a-e5221f6d9fce None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:46.386807 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca40ff-5767-4454-a08a-e5221f6d9fce None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:46.420990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293b0be-fd64-42e5-abd1-bc0eeb3fe352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:46.421555 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293b0be-fd64-42e5-abd1-bc0eeb3fe352 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:46.421832 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293b0be-fd64-42e5-abd1-bc0eeb3fe352 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:46.422103 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293b0be-fd64-42e5-abd1-bc0eeb3fe352 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:46.495652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293b0be-fd64-42e5-abd1-bc0eeb3fe352 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:46.497127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293b0be-fd64-42e5-abd1-bc0eeb3fe352 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:46.592769 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 572/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:35:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 09:35:46.601888 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c4195-c0a7-4d61-9f14-a73ecbd69858 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:46.696088 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c4195-c0a7-4d61-9f14-a73ecbd69858 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:46.728869 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a43e56-4cdd-4233-b415-8cee30f62f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:46.729634 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a43e56-4cdd-4233-b415-8cee30f62f27 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:46.729909 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a43e56-4cdd-4233-b415-8cee30f62f27 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:46.730175 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a43e56-4cdd-4233-b415-8cee30f62f27 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:46.737000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a43e56-4cdd-4233-b415-8cee30f62f27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:46.738433 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a43e56-4cdd-4233-b415-8cee30f62f27 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:46.752608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-b1a43e56-4cdd-4233-b415-8cee30f62f27 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 09 09:35:46.752608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:46.752608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 09 09:35:46.752608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 09 09:35:46.752608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:35:46.752608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:35:46.752608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:35:46.752608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:35:46.752608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 09 09:35:46.752608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:46.752608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:35:46.752608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:46.752608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:46.752608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:46.752608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:46.752608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:46.752608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:46.753812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:46.753812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:46.753812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:46.753812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:46.753812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:46.753812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:46.753812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:35:46.753812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:35:46.753812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:35:46.753812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:35:46.753812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:46.753812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:46.753812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:46.753812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:46.753812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:46.753812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:46.754895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:46.754895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:35:46.754895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:46.754895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:46.754895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:46.754895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:46.754895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:46.754895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:46.754895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:46.754895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:46.754895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 09 09:35:46.754895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:35:46.754895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 09 09:35:46.754895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:35:46.754895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 593/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Mon May 9 09:35:46 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:35:46.759946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293b0be-fd64-42e5-abd1-bc0eeb3fe352 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:46.860596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293b0be-fd64-42e5-abd1-bc0eeb3fe352 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:46.894225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f154b-63e2-4300-b49d-424004c771e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:46.894858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f154b-63e2-4300-b49d-424004c771e1 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:46.895132 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f154b-63e2-4300-b49d-424004c771e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:46.895399 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f154b-63e2-4300-b49d-424004c771e1 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:46.896105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f154b-63e2-4300-b49d-424004c771e1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:46.896412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f154b-63e2-4300-b49d-424004c771e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:46.897808 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f154b-63e2-4300-b49d-424004c771e1 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:46.904861 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 573/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Mon May 9 09:35:46 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:49.101430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f25e7-d3b5-4643-afbf-ac2f43d993af None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:49.102002 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f25e7-d3b5-4643-afbf-ac2f43d993af None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:49.102002 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f25e7-d3b5-4643-afbf-ac2f43d993af None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:49.103330 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 594/1167] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:49.109806 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b9243-68ed-449e-8992-6533a376f3f3 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:49.110598 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b9243-68ed-449e-8992-6533a376f3f3 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:49.110772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b9243-68ed-449e-8992-6533a376f3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:49.136015 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-182b9243-68ed-449e-8992-6533a376f3f3 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:49.137234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-182b9243-68ed-449e-8992-6533a376f3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:49.205090 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 574/1168] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:49.241078 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f25e7-d3b5-4643-afbf-ac2f43d993af None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:49.359177 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f25e7-d3b5-4643-afbf-ac2f43d993af None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:49.393032 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294b79a-5b3d-4350-9b53-782a9d535fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:49.393669 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294b79a-5b3d-4350-9b53-782a9d535fc2 None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:49.393938 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294b79a-5b3d-4350-9b53-782a9d535fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:49.394206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294b79a-5b3d-4350-9b53-782a9d535fc2 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:49.394933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294b79a-5b3d-4350-9b53-782a9d535fc2 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:49.396321 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294b79a-5b3d-4350-9b53-782a9d535fc2 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:49.420128 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 595/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Mon May 9 09:35:49 2022] POST /identity/v3/domains => generated 267 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:35:51.629329 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc9fb6-e1e2-48e7-b7fb-d97318c9122a None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:51.630193 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc9fb6-e1e2-48e7-b7fb-d97318c9122a None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:51.630193 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc9fb6-e1e2-48e7-b7fb-d97318c9122a None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:51.632446 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 575/1170] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:51.638912 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9906e996-d331-4782-ba56-02728e404c02 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:51.639711 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9906e996-d331-4782-ba56-02728e404c02 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:51.639868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9906e996-d331-4782-ba56-02728e404c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:51.660574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-9906e996-d331-4782-ba56-02728e404c02 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:51.661916 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-9906e996-d331-4782-ba56-02728e404c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:51.730085 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 596/1171] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:51.756845 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc9fb6-e1e2-48e7-b7fb-d97318c9122a None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:51.854756 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc9fb6-e1e2-48e7-b7fb-d97318c9122a None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:51.887228 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0fced-6938-4f11-9619-4dea5427ff2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:51.888112 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0fced-6938-4f11-9619-4dea5427ff2e None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:51.888389 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0fced-6938-4f11-9619-4dea5427ff2e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:51.888676 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0fced-6938-4f11-9619-4dea5427ff2e None admin] PATH_INFO: `/v3/domains/67d563c985d343cbacf4a56f4ebd4eab` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:51.899084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0fced-6938-4f11-9619-4dea5427ff2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=67d563c985d343cbacf4a56f4ebd4eab)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:51.900530 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0fced-6938-4f11-9619-4dea5427ff2e None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:51.902202 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 576/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Mon May 9 09:35:51 2022] GET /identity/v3/domains/67d563c985d343cbacf4a56f4ebd4eab => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:51.908199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9906e996-d331-4782-ba56-02728e404c02 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:52.007620 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9906e996-d331-4782-ba56-02728e404c02 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:52.040926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70563dc2-cc12-454d-a116-873d2912cf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:52.041558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70563dc2-cc12-454d-a116-873d2912cf6b None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:52.041837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70563dc2-cc12-454d-a116-873d2912cf6b None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:52.042093 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70563dc2-cc12-454d-a116-873d2912cf6b None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:52.042779 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70563dc2-cc12-454d-a116-873d2912cf6b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:52.044226 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70563dc2-cc12-454d-a116-873d2912cf6b None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:52.071687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 597/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Mon May 9 09:35:51 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:35:54.302458 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eed099-fe39-458e-90ed-f056e7433b52 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:54.303370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eed099-fe39-458e-90ed-f056e7433b52 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:54.303370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eed099-fe39-458e-90ed-f056e7433b52 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:54.305614 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 577/1174] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:54.312255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24087ab-06a0-4559-baf2-302cdea2d9ef None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:54.313078 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24087ab-06a0-4559-baf2-302cdea2d9ef None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:54.313243 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24087ab-06a0-4559-baf2-302cdea2d9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:54.332318 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-c24087ab-06a0-4559-baf2-302cdea2d9ef None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:54.333536 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c24087ab-06a0-4559-baf2-302cdea2d9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:54.399631 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 598/1175] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:54.429975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eed099-fe39-458e-90ed-f056e7433b52 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:54.549467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eed099-fe39-458e-90ed-f056e7433b52 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:54.582809 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16f64d-6911-41c0-ac7f-b45c2b14b6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:54.583723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16f64d-6911-41c0-ac7f-b45c2b14b6e5 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:54.583959 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16f64d-6911-41c0-ac7f-b45c2b14b6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:54.584255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16f64d-6911-41c0-ac7f-b45c2b14b6e5 None admin] PATH_INFO: `/v3/domains/67d563c985d343cbacf4a56f4ebd4eab` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:54.585666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16f64d-6911-41c0-ac7f-b45c2b14b6e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=67d563c985d343cbacf4a56f4ebd4eab)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:54.587110 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16f64d-6911-41c0-ac7f-b45c2b14b6e5 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:54.588715 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 578/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Mon May 9 09:35:54 2022] GET /identity/v3/domains/67d563c985d343cbacf4a56f4ebd4eab => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:54.594763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24087ab-06a0-4559-baf2-302cdea2d9ef None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:54.697705 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24087ab-06a0-4559-baf2-302cdea2d9ef None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:54.735425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42439289-95b3-4073-9e36-48ad376389d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:54.736081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42439289-95b3-4073-9e36-48ad376389d8 None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:54.736352 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42439289-95b3-4073-9e36-48ad376389d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:54.736614 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42439289-95b3-4073-9e36-48ad376389d8 None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:54.737354 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42439289-95b3-4073-9e36-48ad376389d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:54.738776 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42439289-95b3-4073-9e36-48ad376389d8 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:54.759946 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 599/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Mon May 9 09:35:54 2022] POST /identity/v3/users => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:35:57.007322 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f549d3-efdf-4eb8-865b-04d2f90928dc None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:57.008139 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f549d3-efdf-4eb8-865b-04d2f90928dc None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:57.008139 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f549d3-efdf-4eb8-865b-04d2f90928dc None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:57.010534 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 579/1178] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:35:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:35:57.016956 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ab652-f1f1-4c34-a888-36ceb7c676b7 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:57.017422 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ab652-f1f1-4c34-a888-36ceb7c676b7 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:57.017524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ab652-f1f1-4c34-a888-36ceb7c676b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:57.037179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-7e6ab652-f1f1-4c34-a888-36ceb7c676b7 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:35:57.038365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-7e6ab652-f1f1-4c34-a888-36ceb7c676b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:35:57.106033 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 600/1179] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:35:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:35:57.138881 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f549d3-efdf-4eb8-865b-04d2f90928dc None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:57.246743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f549d3-efdf-4eb8-865b-04d2f90928dc None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:57.280758 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835faf6f-e6d0-43a3-aa53-4bcc0ddcf440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:57.281477 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835faf6f-e6d0-43a3-aa53-4bcc0ddcf440 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:57.281751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835faf6f-e6d0-43a3-aa53-4bcc0ddcf440 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:57.282018 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835faf6f-e6d0-43a3-aa53-4bcc0ddcf440 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:57.292913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835faf6f-e6d0-43a3-aa53-4bcc0ddcf440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:35:57.300302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-835faf6f-e6d0-43a3-aa53-4bcc0ddcf440 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:35:57.300302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:35:57.300302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:35:57.300302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:35:57.300302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:35:57.300302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:35:57.300302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:35:57.300302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:35:57.300302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:35:57.300302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:35:57.300302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:35:57.300302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:35:57.300302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:35:57.300302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:35:57.300302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:35:57.300302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:35:57.301490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:35:57.301490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:35:57.301490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:35:57.301490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:35:57.301490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:35:57.301490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:35:57.301490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:35:57.301490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:35:57.301490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:35:57.301490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:35:57.301490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:35:57.301490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:35:57.301490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:35:57.301490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:35:57.301490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:35:57.301490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:35:57.302766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:35:57.302766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:35:57.302766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:35:57.302766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:35:57.302766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:35:57.302766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:35:57.302766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:35:57.302766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:35:57.302766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:35:57.302766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:35:57.302766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:35:57.302766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:35:57.302766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 580/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:35:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:35:57.307629 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ab652-f1f1-4c34-a888-36ceb7c676b7 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:57.408195 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ab652-f1f1-4c34-a888-36ceb7c676b7 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:57.441744 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9147e38-b036-45bc-921c-19885d7b3dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:57.442393 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9147e38-b036-45bc-921c-19885d7b3dd6 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:57.442669 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9147e38-b036-45bc-921c-19885d7b3dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:57.442955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9147e38-b036-45bc-921c-19885d7b3dd6 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:57.443702 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9147e38-b036-45bc-921c-19885d7b3dd6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:57.444005 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9147e38-b036-45bc-921c-19885d7b3dd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:57.445402 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9147e38-b036-45bc-921c-19885d7b3dd6 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:57.453302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 601/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:35:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:35:57.459210 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835faf6f-e6d0-43a3-aa53-4bcc0ddcf440 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:57.574289 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835faf6f-e6d0-43a3-aa53-4bcc0ddcf440 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:57.612707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764db40-1a27-4ae8-9299-e411e7fc500e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:57.613578 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764db40-1a27-4ae8-9299-e411e7fc500e None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:57.613853 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764db40-1a27-4ae8-9299-e411e7fc500e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:57.614118 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764db40-1a27-4ae8-9299-e411e7fc500e None admin] PATH_INFO: `/v3/users/bcae79a0c9a44e938534b2ba2eb9142f` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:57.627673 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764db40-1a27-4ae8-9299-e411e7fc500e None admin] RBAC: Authorizing `identity:get_user(user_id=bcae79a0c9a44e938534b2ba2eb9142f)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:57.629160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764db40-1a27-4ae8-9299-e411e7fc500e None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:57.630808 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 581/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:35:57 2022] GET /identity/v3/users/bcae79a0c9a44e938534b2ba2eb9142f => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:57.637506 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9147e38-b036-45bc-921c-19885d7b3dd6 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:57.743503 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9147e38-b036-45bc-921c-19885d7b3dd6 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:57.780300 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e43da73-254f-47cf-bd58-d7a0580ebdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:57.780961 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e43da73-254f-47cf-bd58-d7a0580ebdd6 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:57.781241 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e43da73-254f-47cf-bd58-d7a0580ebdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:57.781510 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e43da73-254f-47cf-bd58-d7a0580ebdd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:57.852270 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e43da73-254f-47cf-bd58-d7a0580ebdd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:57.853673 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e43da73-254f-47cf-bd58-d7a0580ebdd6 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:57.948559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 602/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:35:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 09:35:57.954879 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764db40-1a27-4ae8-9299-e411e7fc500e None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:58.054403 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764db40-1a27-4ae8-9299-e411e7fc500e None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:58.088336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399d374-fceb-4d6c-97ba-139b1892e44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:58.089172 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399d374-fceb-4d6c-97ba-139b1892e44c None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:58.089470 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399d374-fceb-4d6c-97ba-139b1892e44c None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:58.089736 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399d374-fceb-4d6c-97ba-139b1892e44c None admin] PATH_INFO: `/v3/projects/54fe2a26963b4bcfa84252d49a3f086b` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:58.090953 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b399d374-fceb-4d6c-97ba-139b1892e44c None admin] RBAC: Authorizing `identity:get_project(project_id=54fe2a26963b4bcfa84252d49a3f086b)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:58.092381 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b399d374-fceb-4d6c-97ba-139b1892e44c None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:58.094105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 582/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:35:57 2022] GET /identity/v3/projects/54fe2a26963b4bcfa84252d49a3f086b => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:35:58.100909 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e43da73-254f-47cf-bd58-d7a0580ebdd6 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:35:58.209945 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e43da73-254f-47cf-bd58-d7a0580ebdd6 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:35:58.244231 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e550fe8-3e84-4345-a21f-63ad1c5ca7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:35:58.244897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e550fe8-3e84-4345-a21f-63ad1c5ca7ab None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:35:58.245175 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e550fe8-3e84-4345-a21f-63ad1c5ca7ab None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:35:58.245445 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e550fe8-3e84-4345-a21f-63ad1c5ca7ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:35:58.246272 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e550fe8-3e84-4345-a21f-63ad1c5ca7ab None admin] RBAC: Adding query filter params (role.id=70ac549382b14b2cbb8b35e5dfc648d4, scope.project.id=54fe2a26963b4bcfa84252d49a3f086b, user.id=bcae79a0c9a44e938534b2ba2eb9142f) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:35:58.246584 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e550fe8-3e84-4345-a21f-63ad1c5ca7ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:35:58.248032 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e550fe8-3e84-4345-a21f-63ad1c5ca7ab None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:35:58.253547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 603/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:35:58 2022] GET /identity/v3/role_assignments?user.id=bcae79a0c9a44e938534b2ba2eb9142f&scope.project.id=54fe2a26963b4bcfa84252d49a3f086b&role.id=70ac549382b14b2cbb8b35e5dfc648d4 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:36:00.436055 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22925b74-0e41-4630-8b73-b9559694a8f1 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:36:00.436649 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22925b74-0e41-4630-8b73-b9559694a8f1 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:36:00.436649 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22925b74-0e41-4630-8b73-b9559694a8f1 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:36:00.438024 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 583/1186] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:36:00.443499 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9c9a8-5371-48f9-b731-4b8205c75289 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:36:00.443956 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9c9a8-5371-48f9-b731-4b8205c75289 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:36:00.444048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9c9a8-5371-48f9-b731-4b8205c75289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:36:00.462399 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-c4e9c9a8-5371-48f9-b731-4b8205c75289 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:36:00.463584 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c4e9c9a8-5371-48f9-b731-4b8205c75289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:36:00.529962 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 604/1187] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:36:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:36:00.552940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22925b74-0e41-4630-8b73-b9559694a8f1 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:36:00.659609 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22925b74-0e41-4630-8b73-b9559694a8f1 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:36:00.699514 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedaa685-e5fe-437a-886a-e589c4a7d2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:36:00.700538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedaa685-e5fe-437a-886a-e589c4a7d2b4 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:36:00.700941 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedaa685-e5fe-437a-886a-e589c4a7d2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:36:00.701276 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedaa685-e5fe-437a-886a-e589c4a7d2b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:36:00.714967 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedaa685-e5fe-437a-886a-e589c4a7d2b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:36:00.722546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-fedaa685-e5fe-437a-886a-e589c4a7d2b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:36:00.722546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:36:00.722546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:36:00.722546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:36:00.722546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:36:00.722546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:36:00.722546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:36:00.722546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:36:00.722546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:36:00.722546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:36:00.722546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:36:00.722546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:36:00.722546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:36:00.722546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:36:00.722546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:36:00.722546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:36:00.723667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:36:00.723667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:36:00.723667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:36:00.723667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:36:00.723667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:36:00.723667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:36:00.723667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:36:00.723667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:36:00.723667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:36:00.723667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:36:00.723667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:36:00.723667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:36:00.723667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:36:00.723667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:36:00.723667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:36:00.723667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:36:00.724597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:36:00.724597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:36:00.724597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:36:00.724597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:36:00.724597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:36:00.724597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:36:00.724597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:36:00.724597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:36:00.724597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:36:00.724597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:36:00.724597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:36:00.724597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:36:00.724597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 584/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:36:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:36:00.729959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9c9a8-5371-48f9-b731-4b8205c75289 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:36:00.838513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9c9a8-5371-48f9-b731-4b8205c75289 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:36:00.872796 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96070d6c-84d1-401e-bd4f-965cb9187600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:36:00.873446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96070d6c-84d1-401e-bd4f-965cb9187600 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:36:00.873721 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96070d6c-84d1-401e-bd4f-965cb9187600 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:36:00.873992 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96070d6c-84d1-401e-bd4f-965cb9187600 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:36:00.874738 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96070d6c-84d1-401e-bd4f-965cb9187600 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:36:00.875039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96070d6c-84d1-401e-bd4f-965cb9187600 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:36:00.876417 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96070d6c-84d1-401e-bd4f-965cb9187600 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:36:00.884416 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 605/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:36:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:36:00.890457 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedaa685-e5fe-437a-886a-e589c4a7d2b4 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:36:00.987800 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedaa685-e5fe-437a-886a-e589c4a7d2b4 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:36:01.022002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c59ba8-c26f-4010-b5fd-b82fb3ca6f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:36:01.022842 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c59ba8-c26f-4010-b5fd-b82fb3ca6f3b None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:36:01.023119 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c59ba8-c26f-4010-b5fd-b82fb3ca6f3b None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:36:01.023382 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c59ba8-c26f-4010-b5fd-b82fb3ca6f3b None admin] PATH_INFO: `/v3/users/bcae79a0c9a44e938534b2ba2eb9142f` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:36:01.024709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c59ba8-c26f-4010-b5fd-b82fb3ca6f3b None admin] RBAC: Authorizing `identity:get_user(user_id=bcae79a0c9a44e938534b2ba2eb9142f)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:36:01.026139 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c59ba8-c26f-4010-b5fd-b82fb3ca6f3b None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:36:01.027739 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 585/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:36:00 2022] GET /identity/v3/users/bcae79a0c9a44e938534b2ba2eb9142f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:36:01.033616 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96070d6c-84d1-401e-bd4f-965cb9187600 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:36:01.132684 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96070d6c-84d1-401e-bd4f-965cb9187600 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:36:01.166506 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a91a645-6b56-4b89-b39f-aa1d53d90000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:36:01.167341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91a645-6b56-4b89-b39f-aa1d53d90000 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:36:01.167618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91a645-6b56-4b89-b39f-aa1d53d90000 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:36:01.167891 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91a645-6b56-4b89-b39f-aa1d53d90000 None admin] PATH_INFO: `/v3/projects/54fe2a26963b4bcfa84252d49a3f086b` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:36:01.169228 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a91a645-6b56-4b89-b39f-aa1d53d90000 None admin] RBAC: Authorizing `identity:get_project(project_id=54fe2a26963b4bcfa84252d49a3f086b)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:36:01.170679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a91a645-6b56-4b89-b39f-aa1d53d90000 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:36:01.172373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 606/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:36:01 2022] GET /identity/v3/projects/54fe2a26963b4bcfa84252d49a3f086b => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:36:01.178557 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c59ba8-c26f-4010-b5fd-b82fb3ca6f3b None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:36:01.279968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c59ba8-c26f-4010-b5fd-b82fb3ca6f3b None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:36:01.312802 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2509c-7158-4d7f-9ebf-77f2e747278a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:36:01.313673 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2509c-7158-4d7f-9ebf-77f2e747278a None admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:36:01.313951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2509c-7158-4d7f-9ebf-77f2e747278a None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:36:01.314216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2509c-7158-4d7f-9ebf-77f2e747278a None admin] PATH_INFO: `/v3/projects/54fe2a26963b4bcfa84252d49a3f086b/users/bcae79a0c9a44e938534b2ba2eb9142f/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:36:01.316214 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b2509c-7158-4d7f-9ebf-77f2e747278a None admin] RBAC: Authorizing `identity:create_grant(project_id=54fe2a26963b4bcfa84252d49a3f086b, user_id=bcae79a0c9a44e938534b2ba2eb9142f, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:36:01.317699 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b2509c-7158-4d7f-9ebf-77f2e747278a None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:36:01.333702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 586/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:36:01 2022] PUT /identity/v3/projects/54fe2a26963b4bcfa84252d49a3f086b/users/bcae79a0c9a44e938534b2ba2eb9142f/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:36:03.495357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2773210-f2e7-4314-9101-317d3c49c690 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:36:03.496229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2773210-f2e7-4314-9101-317d3c49c690 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:36:03.496229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2773210-f2e7-4314-9101-317d3c49c690 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:36:03.498413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 607/1193] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:36:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:36:03.507244 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072fa583-de58-4a5b-b8ab-c6f4415a31eb None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:36:03.507682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072fa583-de58-4a5b-b8ab-c6f4415a31eb None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:36:03.507809 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072fa583-de58-4a5b-b8ab-c6f4415a31eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:36:03.529290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-072fa583-de58-4a5b-b8ab-c6f4415a31eb None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:36:03.530474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-072fa583-de58-4a5b-b8ab-c6f4415a31eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:36:03.601761 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 587/1194] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:36:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:36:03.625391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2773210-f2e7-4314-9101-317d3c49c690 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:36:03.730420 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2773210-f2e7-4314-9101-317d3c49c690 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:36:03.763997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40322f-9b15-4662-916c-0ab8264357a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:36:03.764832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40322f-9b15-4662-916c-0ab8264357a9 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:36:03.765096 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40322f-9b15-4662-916c-0ab8264357a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:36:03.765370 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40322f-9b15-4662-916c-0ab8264357a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:36:03.775140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40322f-9b15-4662-916c-0ab8264357a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:36:03.782732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-4a40322f-9b15-4662-916c-0ab8264357a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:36:03.782732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:36:03.782732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:36:03.782732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:36:03.782732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:36:03.782732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:36:03.782732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:36:03.782732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:36:03.782732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:36:03.782732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:36:03.782732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:36:03.782732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:36:03.782732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:36:03.782732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:36:03.782732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:36:03.782732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:36:03.783837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:36:03.783837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:36:03.783837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:36:03.783837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:36:03.783837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:36:03.783837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:36:03.783837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:36:03.783837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:36:03.783837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:36:03.783837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:36:03.783837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:36:03.783837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:36:03.783837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:36:03.783837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:36:03.783837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:36:03.783837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:36:03.785060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:36:03.785060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:36:03.785060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:36:03.785060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:36:03.785060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:36:03.785060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:36:03.785060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:36:03.785060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:36:03.785060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:36:03.785060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:36:03.785060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:36:03.785060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:36:03.785060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 608/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:36:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:36:03.791839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072fa583-de58-4a5b-b8ab-c6f4415a31eb None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:36:03.910017 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072fa583-de58-4a5b-b8ab-c6f4415a31eb None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:36:03.943644 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6300dcb-ee3f-4f70-b01d-357662552b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:36:03.944299 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6300dcb-ee3f-4f70-b01d-357662552b52 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:36:03.944575 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6300dcb-ee3f-4f70-b01d-357662552b52 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:36:03.944878 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6300dcb-ee3f-4f70-b01d-357662552b52 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:36:03.945625 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6300dcb-ee3f-4f70-b01d-357662552b52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:36:03.945927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6300dcb-ee3f-4f70-b01d-357662552b52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:36:03.947298 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6300dcb-ee3f-4f70-b01d-357662552b52 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:36:03.955275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 588/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:36:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:36:03.964027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40322f-9b15-4662-916c-0ab8264357a9 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:36:04.064897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40322f-9b15-4662-916c-0ab8264357a9 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:36:04.099018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaffad50-2bc4-444e-b6bb-ed301e71e769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:36:04.100361 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffad50-2bc4-444e-b6bb-ed301e71e769 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:36:04.100776 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffad50-2bc4-444e-b6bb-ed301e71e769 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:36:04.101237 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffad50-2bc4-444e-b6bb-ed301e71e769 None admin] PATH_INFO: `/v3/users/bcae79a0c9a44e938534b2ba2eb9142f` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:36:04.103317 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaffad50-2bc4-444e-b6bb-ed301e71e769 None admin] RBAC: Authorizing `identity:get_user(user_id=bcae79a0c9a44e938534b2ba2eb9142f)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:36:04.105743 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaffad50-2bc4-444e-b6bb-ed301e71e769 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:36:04.108333 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 609/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Mon May 9 09:36:03 2022] GET /identity/v3/users/bcae79a0c9a44e938534b2ba2eb9142f => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:36:04.117271 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6300dcb-ee3f-4f70-b01d-357662552b52 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:36:04.226393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6300dcb-ee3f-4f70-b01d-357662552b52 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:36:04.263792 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b095a9-36e0-4602-9872-577c900e1503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:36:04.264419 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b095a9-36e0-4602-9872-577c900e1503 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:36:04.264703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b095a9-36e0-4602-9872-577c900e1503 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:36:04.264989 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b095a9-36e0-4602-9872-577c900e1503 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:36:04.331695 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b095a9-36e0-4602-9872-577c900e1503 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:36:04.333150 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b095a9-36e0-4602-9872-577c900e1503 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:36:04.428815 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 589/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:36:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 09:36:04.441365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaffad50-2bc4-444e-b6bb-ed301e71e769 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:36:04.546671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaffad50-2bc4-444e-b6bb-ed301e71e769 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:36:04.585061 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b112ef95-4e77-4c46-a979-769dbc1b44f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:36:04.585857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112ef95-4e77-4c46-a979-769dbc1b44f7 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:36:04.586127 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112ef95-4e77-4c46-a979-769dbc1b44f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:36:04.586394 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112ef95-4e77-4c46-a979-769dbc1b44f7 None admin] PATH_INFO: `/v3/projects/54fe2a26963b4bcfa84252d49a3f086b` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:36:04.587625 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b112ef95-4e77-4c46-a979-769dbc1b44f7 None admin] RBAC: Authorizing `identity:get_project(project_id=54fe2a26963b4bcfa84252d49a3f086b)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:36:04.589098 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b112ef95-4e77-4c46-a979-769dbc1b44f7 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:36:04.590787 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 610/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Mon May 9 09:36:04 2022] GET /identity/v3/projects/54fe2a26963b4bcfa84252d49a3f086b => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:36:04.597132 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b095a9-36e0-4602-9872-577c900e1503 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:36:04.703533 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b095a9-36e0-4602-9872-577c900e1503 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:36:04.740941 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69667e9f-b343-471b-bbfe-c600edd0f8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:36:04.741569 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69667e9f-b343-471b-bbfe-c600edd0f8cc None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:36:04.741845 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69667e9f-b343-471b-bbfe-c600edd0f8cc None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:36:04.742106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69667e9f-b343-471b-bbfe-c600edd0f8cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:36:04.743287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69667e9f-b343-471b-bbfe-c600edd0f8cc None admin] RBAC: Adding query filter params (role.id=70ac549382b14b2cbb8b35e5dfc648d4, scope.project.id=54fe2a26963b4bcfa84252d49a3f086b, user.id=bcae79a0c9a44e938534b2ba2eb9142f) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:36:04.743590 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69667e9f-b343-471b-bbfe-c600edd0f8cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:36:04.745035 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69667e9f-b343-471b-bbfe-c600edd0f8cc None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:36:04.750746 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 590/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:36:04 2022] GET /identity/v3/role_assignments?user.id=bcae79a0c9a44e938534b2ba2eb9142f&scope.project.id=54fe2a26963b4bcfa84252d49a3f086b&role.id=70ac549382b14b2cbb8b35e5dfc648d4 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:05.293839 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7b997-f311-4f11-8b7a-c510461b79e3 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:05.294467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7b997-f311-4f11-8b7a-c510461b79e3 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:05.294650 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7b997-f311-4f11-8b7a-c510461b79e3 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:05.296684 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 611/1201] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:37:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:37:05.305201 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3981a31-036c-4a07-9807-010c0cb30a97 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:05.305201 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3981a31-036c-4a07-9807-010c0cb30a97 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:05.305201 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3981a31-036c-4a07-9807-010c0cb30a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:05.322731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-a3981a31-036c-4a07-9807-010c0cb30a97 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:37:05.323991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a3981a31-036c-4a07-9807-010c0cb30a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:37:05.400159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 591/1202] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:37:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:37:05.429947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7b997-f311-4f11-8b7a-c510461b79e3 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:05.541727 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7b997-f311-4f11-8b7a-c510461b79e3 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:05.584162 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ad97b-8a9e-4b47-a3c2-7e20a8e50911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:05.586061 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ad97b-8a9e-4b47-a3c2-7e20a8e50911 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:05.586413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ad97b-8a9e-4b47-a3c2-7e20a8e50911 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:05.586805 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ad97b-8a9e-4b47-a3c2-7e20a8e50911 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:05.599425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ad97b-8a9e-4b47-a3c2-7e20a8e50911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:05.601404 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ad97b-8a9e-4b47-a3c2-7e20a8e50911 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:05.620868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-d06ad97b-8a9e-4b47-a3c2-7e20a8e50911 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:05.620868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:05.620868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:05.620868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:05.620868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:05.620868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:05.620868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:05.620868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:05.620868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:05.620868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:05.620868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:05.620868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:05.620868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:37:05.620868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:37:05.620868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:37:05.622146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:37:05.622146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:05.622146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:05.622146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:05.622146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:05.622146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:05.622146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:05.622146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:05.622146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:37:05.622146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:05.622146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:05.622146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:05.622146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:05.622146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:05.622146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:05.622146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:05.622146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:05.628075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:37:05.628075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:37:05.628075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:05.628075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:37:05.628075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 612/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:37:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:05.628396 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3981a31-036c-4a07-9807-010c0cb30a97 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:05.754334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3981a31-036c-4a07-9807-010c0cb30a97 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:05.796501 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8ec19-fab7-4073-9515-946942ac0d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:05.797202 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8ec19-fab7-4073-9515-946942ac0d0d None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:05.797516 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8ec19-fab7-4073-9515-946942ac0d0d None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:05.797781 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8ec19-fab7-4073-9515-946942ac0d0d None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:05.798579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8ec19-fab7-4073-9515-946942ac0d0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:05.798878 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8ec19-fab7-4073-9515-946942ac0d0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:05.800241 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8ec19-fab7-4073-9515-946942ac0d0d None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:05.813052 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 592/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:05.818886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ad97b-8a9e-4b47-a3c2-7e20a8e50911 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:05.930065 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ad97b-8a9e-4b47-a3c2-7e20a8e50911 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:05.963850 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd7433-8d6e-4ec7-9d48-b6373ba014ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:05.964565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd7433-8d6e-4ec7-9d48-b6373ba014ba None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:05.964876 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd7433-8d6e-4ec7-9d48-b6373ba014ba None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:05.965143 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd7433-8d6e-4ec7-9d48-b6373ba014ba None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:05.965904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd7433-8d6e-4ec7-9d48-b6373ba014ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:05.967376 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd7433-8d6e-4ec7-9d48-b6373ba014ba None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:05.996907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 613/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Mon May 9 09:37:05 2022] POST /identity/v3/users => generated 251 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:37:08.455808 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433bf41-5d8c-4e0a-acec-9d277ad06306 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:08.458353 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433bf41-5d8c-4e0a-acec-9d277ad06306 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:08.458353 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433bf41-5d8c-4e0a-acec-9d277ad06306 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:08.458353 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 593/1206] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:37:08.464437 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd497c5-f3bf-4aa0-9f33-03cd652a678e None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:08.465254 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd497c5-f3bf-4aa0-9f33-03cd652a678e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:08.465541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd497c5-f3bf-4aa0-9f33-03cd652a678e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:08.484835 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-6fd497c5-f3bf-4aa0-9f33-03cd652a678e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:37:08.486278 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-6fd497c5-f3bf-4aa0-9f33-03cd652a678e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:37:08.561118 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 614/1207] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:37:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:37:08.585405 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433bf41-5d8c-4e0a-acec-9d277ad06306 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:08.699691 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433bf41-5d8c-4e0a-acec-9d277ad06306 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:08.734931 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792b14b-2dea-480e-9663-762b05c686cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:08.735703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792b14b-2dea-480e-9663-762b05c686cd None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:08.736006 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792b14b-2dea-480e-9663-762b05c686cd None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:08.736242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792b14b-2dea-480e-9663-762b05c686cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:08.747494 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792b14b-2dea-480e-9663-762b05c686cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:37:08.755088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-e792b14b-2dea-480e-9663-762b05c686cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 09:37:08.755088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:08.755088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:08.755088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:08.755088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:08.755088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:08.755088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:08.755088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:08.755088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:08.755088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:08.755088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:08.755088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:08.755088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:37:08.755088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:37:08.755088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:37:08.755088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:37:08.756216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:37:08.756216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:37:08.756216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:37:08.756216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:37:08.756216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:08.756216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:08.756216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:08.756216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:08.756216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:08.756216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:08.756216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:08.756216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:37:08.756216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:08.756216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:08.756216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:08.756216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:08.758793 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:08.758793 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:08.758793 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:08.758793 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:08.758793 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:37:08.758793 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:37:08.758793 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:37:08.758793 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:37:08.758793 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:37:08.758793 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:37:08.758793 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 09:37:08.758793 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:08.758793 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 594/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:37:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:08.764074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd497c5-f3bf-4aa0-9f33-03cd652a678e None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:08.870381 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd497c5-f3bf-4aa0-9f33-03cd652a678e None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:08.905532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e918b-aa5c-4280-9f74-a3f643d10d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:08.906208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e918b-aa5c-4280-9f74-a3f643d10d70 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:08.906485 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e918b-aa5c-4280-9f74-a3f643d10d70 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:08.906765 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e918b-aa5c-4280-9f74-a3f643d10d70 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:08.907535 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e918b-aa5c-4280-9f74-a3f643d10d70 None admin] RBAC: Adding query filter params (name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:08.907836 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e918b-aa5c-4280-9f74-a3f643d10d70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:08.909281 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e918b-aa5c-4280-9f74-a3f643d10d70 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:08.918506 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 615/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Mon May 9 09:37:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:08.924514 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792b14b-2dea-480e-9663-762b05c686cd None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:09.033267 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792b14b-2dea-480e-9663-762b05c686cd None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:09.068377 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607c114-7766-40db-9ed0-6dc062903958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:09.069306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607c114-7766-40db-9ed0-6dc062903958 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:09.069580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607c114-7766-40db-9ed0-6dc062903958 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:09.069829 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607c114-7766-40db-9ed0-6dc062903958 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:09.081427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3607c114-7766-40db-9ed0-6dc062903958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:09.082953 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3607c114-7766-40db-9ed0-6dc062903958 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:09.100805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-3607c114-7766-40db-9ed0-6dc062903958 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:09.100805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:09.100805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:09.100805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:09.100805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:09.100805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:09.100805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:09.100805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:09.100805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:09.100805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:09.100805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:09.100805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:09.100805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:37:09.100805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:37:09.100805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:37:09.101958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:37:09.101958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:09.101958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:09.101958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:09.101958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:09.101958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:09.101958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:09.101958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:09.101958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:37:09.101958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:09.101958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:09.101958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:09.101958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:09.101958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:09.101958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:09.101958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:09.101958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:09.103189 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:37:09.103189 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:37:09.103189 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:09.103189 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:09.103189 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 595/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:37:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:37:09.107905 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e918b-aa5c-4280-9f74-a3f643d10d70 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:09.208465 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e918b-aa5c-4280-9f74-a3f643d10d70 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:09.244282 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f3110-5c45-499b-bba5-5b9ec0f766c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:09.244972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f3110-5c45-499b-bba5-5b9ec0f766c9 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:09.245241 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f3110-5c45-499b-bba5-5b9ec0f766c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:09.245505 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f3110-5c45-499b-bba5-5b9ec0f766c9 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:09.246266 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4f3110-5c45-499b-bba5-5b9ec0f766c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:09.246538 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4f3110-5c45-499b-bba5-5b9ec0f766c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:09.247969 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4f3110-5c45-499b-bba5-5b9ec0f766c9 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:09.261425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 616/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:37:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:09.267350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607c114-7766-40db-9ed0-6dc062903958 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:09.380027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607c114-7766-40db-9ed0-6dc062903958 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:09.421051 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c8b6d-5485-407e-bed0-0b98d33e9ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:09.421904 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c8b6d-5485-407e-bed0-0b98d33e9ed8 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:09.422249 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c8b6d-5485-407e-bed0-0b98d33e9ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:09.422555 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c8b6d-5485-407e-bed0-0b98d33e9ed8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:09.431742 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c8b6d-5485-407e-bed0-0b98d33e9ed8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:09.433312 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c8b6d-5485-407e-bed0-0b98d33e9ed8 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:09.450352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-4a2c8b6d-5485-407e-bed0-0b98d33e9ed8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 09:37:09.450352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:09.450352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:09.450352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:09.450352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:09.450352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:09.450352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:09.450352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:09.450352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:09.450352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:09.450352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:09.450352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:09.450352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:37:09.450352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:37:09.450352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:37:09.450352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:37:09.451503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:09.451503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:09.451503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:37:09.451503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:37:09.451503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:37:09.451503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:37:09.451503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:09.451503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:09.451503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:09.451503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:09.451503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:09.451503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:37:09.451503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:09.451503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:09.451503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:09.451503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:09.451503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:09.451503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:09.452469 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:09.452469 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:09.452469 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:37:09.452469 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:37:09.452469 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:37:09.452469 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:37:09.452469 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:37:09.452469 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:37:09.452469 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 09:37:09.452469 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:09.452469 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 596/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:37:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:09.457730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f3110-5c45-499b-bba5-5b9ec0f766c9 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:09.568261 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f3110-5c45-499b-bba5-5b9ec0f766c9 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:09.611666 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97fc47f-bdec-4ee1-83f0-78d155cd8840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:09.612854 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fc47f-bdec-4ee1-83f0-78d155cd8840 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:09.613300 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fc47f-bdec-4ee1-83f0-78d155cd8840 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:09.613618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fc47f-bdec-4ee1-83f0-78d155cd8840 None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:09.614720 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97fc47f-bdec-4ee1-83f0-78d155cd8840 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:09.616337 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97fc47f-bdec-4ee1-83f0-78d155cd8840 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:09.616931 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97fc47f-bdec-4ee1-83f0-78d155cd8840 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:09.640779 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 617/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Mon May 9 09:37:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:09.644226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c8b6d-5485-407e-bed0-0b98d33e9ed8 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:09.763282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c8b6d-5485-407e-bed0-0b98d33e9ed8 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:09.799179 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d472373c-9511-45e4-a522-2009f47f579b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:09.799833 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472373c-9511-45e4-a522-2009f47f579b None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:09.800201 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472373c-9511-45e4-a522-2009f47f579b None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:09.800484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472373c-9511-45e4-a522-2009f47f579b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:09.874285 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d472373c-9511-45e4-a522-2009f47f579b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:09.875777 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d472373c-9511-45e4-a522-2009f47f579b None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:09.980446 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 597/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:37:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 09:37:09.982121 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97fc47f-bdec-4ee1-83f0-78d155cd8840 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:10.094234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97fc47f-bdec-4ee1-83f0-78d155cd8840 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:10.134561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e46fec-5e4a-4862-b9aa-095d0d1e302a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:10.135651 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e46fec-5e4a-4862-b9aa-095d0d1e302a None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:10.136020 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e46fec-5e4a-4862-b9aa-095d0d1e302a None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:10.136456 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e46fec-5e4a-4862-b9aa-095d0d1e302a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:10.148693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e46fec-5e4a-4862-b9aa-095d0d1e302a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:10.150242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e46fec-5e4a-4862-b9aa-095d0d1e302a None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:10.172612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-24e46fec-5e4a-4862-b9aa-095d0d1e302a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:10.172612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:10.172612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:10.172612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:10.172612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:10.172612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:10.172612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:10.172612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:10.172612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:10.172612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:10.172612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:10.172612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:10.172612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:37:10.172612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:37:10.172612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:37:10.173904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:37:10.173904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:10.173904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:10.173904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:10.173904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:10.173904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:10.173904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:10.173904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:10.173904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:37:10.173904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:10.173904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:10.173904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:10.173904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:10.173904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:10.173904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:10.173904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:10.173904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:10.174929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:37:10.174929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:37:10.174929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:10.174929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:37:10.174929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 618/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:37:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:37:10.179978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d472373c-9511-45e4-a522-2009f47f579b None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:10.289489 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d472373c-9511-45e4-a522-2009f47f579b None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:10.330100 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5284f99-1d9b-4b7a-b04c-96e3f276a7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:10.330964 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5284f99-1d9b-4b7a-b04c-96e3f276a7cd None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:10.331246 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5284f99-1d9b-4b7a-b04c-96e3f276a7cd None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:10.331517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5284f99-1d9b-4b7a-b04c-96e3f276a7cd None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:10.332443 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5284f99-1d9b-4b7a-b04c-96e3f276a7cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:10.332773 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5284f99-1d9b-4b7a-b04c-96e3f276a7cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:10.334479 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5284f99-1d9b-4b7a-b04c-96e3f276a7cd None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:10.347682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 598/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:10.353706 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e46fec-5e4a-4862-b9aa-095d0d1e302a None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:10.460621 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e46fec-5e4a-4862-b9aa-095d0d1e302a None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:10.495526 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649d7cb-70cb-4771-8619-ee6957148c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:10.496392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649d7cb-70cb-4771-8619-ee6957148c9d None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:10.496824 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649d7cb-70cb-4771-8619-ee6957148c9d None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:10.497059 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649d7cb-70cb-4771-8619-ee6957148c9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:10.504090 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d649d7cb-70cb-4771-8619-ee6957148c9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:10.505558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d649d7cb-70cb-4771-8619-ee6957148c9d None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:10.521112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-d649d7cb-70cb-4771-8619-ee6957148c9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:37:10.521112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:10.521112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:10.521112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:10.521112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:10.521112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:10.521112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:10.521112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:10.521112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:10.521112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:10.521112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:10.521112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:10.521112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:37:10.521112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:37:10.521112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:37:10.522314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:37:10.522314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:10.522314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:10.522314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:10.522314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:10.522314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:10.522314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:10.522314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:10.522314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:37:10.522314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:10.522314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:10.522314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:10.522314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:10.522314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:10.522314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:10.522314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:10.522314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:10.523484 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:37:10.523484 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:37:10.523484 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:37:10.523484 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:37:10.523484 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:37:10.523484 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:37:10.523484 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:37:10.523484 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:37:10.524645 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 619/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:37:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:37:10.530788 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5284f99-1d9b-4b7a-b04c-96e3f276a7cd None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:10.650176 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5284f99-1d9b-4b7a-b04c-96e3f276a7cd None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:10.693622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062a06e-7731-46ad-8393-163af5802253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:10.694287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062a06e-7731-46ad-8393-163af5802253 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:10.694567 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062a06e-7731-46ad-8393-163af5802253 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:10.694915 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062a06e-7731-46ad-8393-163af5802253 None admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:10.695634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062a06e-7731-46ad-8393-163af5802253 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:10.696039 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062a06e-7731-46ad-8393-163af5802253 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:10.697522 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062a06e-7731-46ad-8393-163af5802253 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:10.716253 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 599/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:37:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:10.718752 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649d7cb-70cb-4771-8619-ee6957148c9d None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:10.828329 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649d7cb-70cb-4771-8619-ee6957148c9d None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:10.870062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46174d-6f9c-4d9e-bce6-7c0ea6d63378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:10.870961 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46174d-6f9c-4d9e-bce6-7c0ea6d63378 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:10.871438 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46174d-6f9c-4d9e-bce6-7c0ea6d63378 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:10.871781 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46174d-6f9c-4d9e-bce6-7c0ea6d63378 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:10.872912 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46174d-6f9c-4d9e-bce6-7c0ea6d63378 None admin] RBAC: Adding query filter params (role.id=f55c3728df78466989d0cd1b13c5ea28, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=6c9e458c2c5f4ed1b3eddd84c2ec1703) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:10.873284 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46174d-6f9c-4d9e-bce6-7c0ea6d63378 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:10.875077 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46174d-6f9c-4d9e-bce6-7c0ea6d63378 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:10.884696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 620/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:37:10 2022] GET /identity/v3/role_assignments?user.id=6c9e458c2c5f4ed1b3eddd84c2ec1703&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=f55c3728df78466989d0cd1b13c5ea28 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:37:13.250195 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a24dc0-405c-48a2-8e39-95522cb23e85 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:13.250817 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a24dc0-405c-48a2-8e39-95522cb23e85 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:13.250817 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a24dc0-405c-48a2-8e39-95522cb23e85 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:13.252263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 600/1220] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:37:13.257746 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bcd68-6f45-42d0-879c-ac679ffb60ae None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:13.258245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bcd68-6f45-42d0-879c-ac679ffb60ae None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:13.258479 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bcd68-6f45-42d0-879c-ac679ffb60ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:13.275832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-e29bcd68-6f45-42d0-879c-ac679ffb60ae None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:37:13.277066 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e29bcd68-6f45-42d0-879c-ac679ffb60ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:37:13.360711 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 621/1221] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:37:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:37:13.385195 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a24dc0-405c-48a2-8e39-95522cb23e85 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:13.489880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a24dc0-405c-48a2-8e39-95522cb23e85 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:13.527762 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461cba44-22d4-4bde-8528-79b85847283f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:13.528623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cba44-22d4-4bde-8528-79b85847283f None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:13.529028 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cba44-22d4-4bde-8528-79b85847283f None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:13.529312 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cba44-22d4-4bde-8528-79b85847283f None admin] PATH_INFO: `/v3/roles/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:13.541568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461cba44-22d4-4bde-8528-79b85847283f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:37:13.549281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-461cba44-22d4-4bde-8528-79b85847283f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 09:37:13.549281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:13.549281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:13.549281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:13.549281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:13.549281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:13.549281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:13.549281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:13.549281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:13.549281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:13.549281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:13.549281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:13.549281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:37:13.549281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:37:13.549281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:37:13.549281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:37:13.550362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:37:13.550362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:37:13.550362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:37:13.550362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:37:13.550362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:13.550362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:13.550362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:13.550362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:13.550362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:13.550362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:13.550362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:13.550362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:37:13.550362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:13.550362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:13.550362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:13.550362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:13.556281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:13.556281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:13.556281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:13.556281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:13.556281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:37:13.556281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:37:13.556281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:37:13.556281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:37:13.556281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:37:13.556281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:37:13.556281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 09:37:13.556281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:13.556281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 601/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:37:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:37:13.558113 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29bcd68-6f45-42d0-879c-ac679ffb60ae None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:13.658814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29bcd68-6f45-42d0-879c-ac679ffb60ae None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:13.700530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90153d55-7a17-4feb-be14-38876ac1ecf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:13.701453 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90153d55-7a17-4feb-be14-38876ac1ecf0 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:13.701925 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90153d55-7a17-4feb-be14-38876ac1ecf0 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:13.702234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90153d55-7a17-4feb-be14-38876ac1ecf0 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:13.703324 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90153d55-7a17-4feb-be14-38876ac1ecf0 None admin] RBAC: Adding query filter params (name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:13.703829 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90153d55-7a17-4feb-be14-38876ac1ecf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:13.705687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90153d55-7a17-4feb-be14-38876ac1ecf0 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:13.716853 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 622/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Mon May 9 09:37:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:13.724291 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461cba44-22d4-4bde-8528-79b85847283f None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:13.833278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461cba44-22d4-4bde-8528-79b85847283f None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:13.868781 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930aa2d4-a9cf-4224-abd8-e071a56eb57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:13.869635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930aa2d4-a9cf-4224-abd8-e071a56eb57c None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:13.869917 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930aa2d4-a9cf-4224-abd8-e071a56eb57c None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:13.870239 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930aa2d4-a9cf-4224-abd8-e071a56eb57c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:13.882160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930aa2d4-a9cf-4224-abd8-e071a56eb57c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:13.883933 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930aa2d4-a9cf-4224-abd8-e071a56eb57c None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:13.903323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-930aa2d4-a9cf-4224-abd8-e071a56eb57c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:13.903323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:13.903323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:13.903323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:13.903323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:13.903323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:13.903323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:13.903323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:13.903323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:13.903323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:13.903323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:13.903323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:13.903323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:37:13.903323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:37:13.903323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:37:13.904433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:37:13.904433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:13.904433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:13.904433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:13.904433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:13.904433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:13.904433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:13.904433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:13.904433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:37:13.904433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:13.904433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:13.904433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:13.904433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:13.904433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:13.904433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:13.904433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:13.904433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:13.905716 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:37:13.905716 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:37:13.905716 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:13.905716 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:13.905716 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 602/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:37:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:13.910710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90153d55-7a17-4feb-be14-38876ac1ecf0 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:14.019251 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90153d55-7a17-4feb-be14-38876ac1ecf0 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:14.054318 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f952c5-46a7-4791-a115-20c4eaf13797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:14.054979 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f952c5-46a7-4791-a115-20c4eaf13797 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:14.055254 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f952c5-46a7-4791-a115-20c4eaf13797 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:14.055521 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f952c5-46a7-4791-a115-20c4eaf13797 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:14.056235 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f952c5-46a7-4791-a115-20c4eaf13797 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:14.056536 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f952c5-46a7-4791-a115-20c4eaf13797 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:14.057997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f952c5-46a7-4791-a115-20c4eaf13797 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:14.075886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 623/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:14.077120 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930aa2d4-a9cf-4224-abd8-e071a56eb57c None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:14.181266 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930aa2d4-a9cf-4224-abd8-e071a56eb57c None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:14.215076 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd959cd-897c-4930-8019-e381800d4ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:14.215968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd959cd-897c-4930-8019-e381800d4ed7 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:14.216249 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd959cd-897c-4930-8019-e381800d4ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:14.216520 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd959cd-897c-4930-8019-e381800d4ed7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:14.224513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd959cd-897c-4930-8019-e381800d4ed7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:14.227354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd959cd-897c-4930-8019-e381800d4ed7 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:14.243433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-9fd959cd-897c-4930-8019-e381800d4ed7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 09:37:14.243433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:14.243433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:14.243433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:14.243433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:14.243433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:14.243433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:14.243433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:14.243433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:14.243433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:14.243433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:14.243433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:14.243433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:37:14.243433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:37:14.243433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:37:14.243433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:37:14.244694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:14.244694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:14.244694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:37:14.244694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:37:14.244694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:37:14.244694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:37:14.244694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:14.244694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:14.244694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:14.244694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:14.244694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:14.244694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:37:14.244694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:14.244694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:14.244694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:14.244694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:14.244694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:14.244694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:14.245810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:14.245810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:14.245810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:37:14.245810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:37:14.245810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:37:14.245810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:37:14.245810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:37:14.245810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:37:14.245810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 09:37:14.245810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:14.245810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 603/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:37:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:37:14.250980 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f952c5-46a7-4791-a115-20c4eaf13797 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:14.359860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f952c5-46a7-4791-a115-20c4eaf13797 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:14.396053 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31e740-7454-409f-a66a-98b334d9b549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:14.396843 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31e740-7454-409f-a66a-98b334d9b549 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:14.397120 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31e740-7454-409f-a66a-98b334d9b549 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:14.397394 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31e740-7454-409f-a66a-98b334d9b549 None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:14.398390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31e740-7454-409f-a66a-98b334d9b549 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:14.398688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31e740-7454-409f-a66a-98b334d9b549 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:14.402181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31e740-7454-409f-a66a-98b334d9b549 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:14.422354 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 624/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Mon May 9 09:37:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:37:14.431232 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd959cd-897c-4930-8019-e381800d4ed7 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:14.536837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd959cd-897c-4930-8019-e381800d4ed7 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:14.571285 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34cb695-867d-45df-98fa-c452761297b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:14.572083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cb695-867d-45df-98fa-c452761297b8 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:14.572357 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cb695-867d-45df-98fa-c452761297b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:14.572620 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cb695-867d-45df-98fa-c452761297b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:14.584091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34cb695-867d-45df-98fa-c452761297b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:14.585752 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34cb695-867d-45df-98fa-c452761297b8 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:14.604282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-f34cb695-867d-45df-98fa-c452761297b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:14.604282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:14.604282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:14.604282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:14.604282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:14.604282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:14.604282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:14.604282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:14.604282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:14.604282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:14.604282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:14.604282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:14.604282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:37:14.604282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:37:14.604282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:37:14.605643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:37:14.605643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:14.605643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:14.605643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:14.605643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:14.605643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:14.605643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:14.605643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:14.605643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:37:14.605643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:14.605643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:14.605643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:14.605643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:14.605643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:14.605643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:14.605643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:14.605643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:14.606987 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:37:14.606987 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:37:14.606987 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:14.606987 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:14.606987 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 604/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:37:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:14.613477 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31e740-7454-409f-a66a-98b334d9b549 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:14.717923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31e740-7454-409f-a66a-98b334d9b549 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:14.758238 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b648af1-eceb-463b-8726-7de3ad82b05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:14.759020 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b648af1-eceb-463b-8726-7de3ad82b05f None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:14.759252 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b648af1-eceb-463b-8726-7de3ad82b05f None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:14.759567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b648af1-eceb-463b-8726-7de3ad82b05f None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:14.760456 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b648af1-eceb-463b-8726-7de3ad82b05f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:14.760865 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b648af1-eceb-463b-8726-7de3ad82b05f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:14.762325 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b648af1-eceb-463b-8726-7de3ad82b05f None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:14.779478 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 625/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:14.781168 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34cb695-867d-45df-98fa-c452761297b8 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:14.887712 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34cb695-867d-45df-98fa-c452761297b8 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:14.922475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b17ff5-5ee5-49c9-982f-c84563e1b9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:14.923385 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b17ff5-5ee5-49c9-982f-c84563e1b9bf None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:14.923728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b17ff5-5ee5-49c9-982f-c84563e1b9bf None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:14.923999 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b17ff5-5ee5-49c9-982f-c84563e1b9bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:14.931026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b17ff5-5ee5-49c9-982f-c84563e1b9bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:14.932480 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b17ff5-5ee5-49c9-982f-c84563e1b9bf None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:14.949109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-91b17ff5-5ee5-49c9-982f-c84563e1b9bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:37:14.949109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:14.949109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:14.949109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:14.949109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:14.949109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:14.949109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:14.949109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:14.949109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:14.949109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:14.949109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:14.949109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:14.949109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:37:14.949109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:37:14.949109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:37:14.950238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:37:14.950238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:14.950238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:14.950238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:14.950238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:14.950238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:14.950238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:14.950238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:14.950238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:37:14.950238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:14.950238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:14.950238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:14.950238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:14.950238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:14.950238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:14.950238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:14.950238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:14.951554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:37:14.951554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:37:14.951554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:37:14.951554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:37:14.951554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:37:14.951554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:37:14.951554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:37:14.951554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:14.951554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 605/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:37:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:14.956463 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b648af1-eceb-463b-8726-7de3ad82b05f None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:15.056701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b648af1-eceb-463b-8726-7de3ad82b05f None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:15.093222 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6651ebd-b33c-475d-bef8-806961e457d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:15.093888 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6651ebd-b33c-475d-bef8-806961e457d5 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:15.094251 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6651ebd-b33c-475d-bef8-806961e457d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:15.094670 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6651ebd-b33c-475d-bef8-806961e457d5 None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:15.095556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6651ebd-b33c-475d-bef8-806961e457d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:15.095907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6651ebd-b33c-475d-bef8-806961e457d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:15.097529 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6651ebd-b33c-475d-bef8-806961e457d5 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:15.110581 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 626/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:37:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:37:15.116988 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b17ff5-5ee5-49c9-982f-c84563e1b9bf None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:15.222323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b17ff5-5ee5-49c9-982f-c84563e1b9bf None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:15.260073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c59f94-fab1-4506-9b6f-1894f6519fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:15.260996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c59f94-fab1-4506-9b6f-1894f6519fa4 None admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:15.261267 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c59f94-fab1-4506-9b6f-1894f6519fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:15.261529 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c59f94-fab1-4506-9b6f-1894f6519fa4 None admin] PATH_INFO: `/v3/projects/9cd39464055141b8b43fde20d0885c99/users/6c9e458c2c5f4ed1b3eddd84c2ec1703/roles/f55c3728df78466989d0cd1b13c5ea28` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:15.276923 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c59f94-fab1-4506-9b6f-1894f6519fa4 None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd39464055141b8b43fde20d0885c99, user_id=6c9e458c2c5f4ed1b3eddd84c2ec1703, role_id=f55c3728df78466989d0cd1b13c5ea28)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:15.278405 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c59f94-fab1-4506-9b6f-1894f6519fa4 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:15.292191 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 606/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:37:15 2022] PUT /identity/v3/projects/9cd39464055141b8b43fde20d0885c99/users/6c9e458c2c5f4ed1b3eddd84c2ec1703/roles/f55c3728df78466989d0cd1b13c5ea28 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:37:17.498184 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b5c7b-7196-4b91-b236-00ac5cc17e29 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:17.498760 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b5c7b-7196-4b91-b236-00ac5cc17e29 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:17.498760 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b5c7b-7196-4b91-b236-00ac5cc17e29 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:17.500129 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 627/1233] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:37:17.505584 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6b586-4917-4f32-a70d-d97abaac3342 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:17.506034 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6b586-4917-4f32-a70d-d97abaac3342 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:17.506143 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6b586-4917-4f32-a70d-d97abaac3342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:17.522439 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-26d6b586-4917-4f32-a70d-d97abaac3342 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:37:17.523629 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-26d6b586-4917-4f32-a70d-d97abaac3342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:37:17.593997 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 607/1234] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:37:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:37:17.617868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b5c7b-7196-4b91-b236-00ac5cc17e29 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:17.743079 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b5c7b-7196-4b91-b236-00ac5cc17e29 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:17.780558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0d74c-8022-4de3-8516-3d86c17a9f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:17.781390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0d74c-8022-4de3-8516-3d86c17a9f69 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:17.781701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0d74c-8022-4de3-8516-3d86c17a9f69 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:17.782028 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0d74c-8022-4de3-8516-3d86c17a9f69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:17.793479 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a0d74c-8022-4de3-8516-3d86c17a9f69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:37:17.803749 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-d7a0d74c-8022-4de3-8516-3d86c17a9f69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 09:37:17.803749 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:17.803749 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:17.803749 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:17.803749 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:17.803749 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:17.803749 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:17.803749 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:17.803749 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:17.803749 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:17.803749 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:17.803749 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:17.803749 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:37:17.803749 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:37:17.803749 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:37:17.803749 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:37:17.804847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:37:17.804847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:37:17.804847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:37:17.804847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:37:17.804847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:17.804847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:17.804847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:17.804847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:17.804847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:17.804847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:17.804847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:17.804847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:37:17.804847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:17.804847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:17.804847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:17.804847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:17.805723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:17.805723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:17.805723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:17.805723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:17.805723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:37:17.805723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:37:17.805723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:37:17.805723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:37:17.805723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:37:17.805723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:37:17.805723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 09:37:17.805723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:37:17.805723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 628/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Mon May 9 09:37:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:17.811092 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6b586-4917-4f32-a70d-d97abaac3342 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:17.920890 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6b586-4917-4f32-a70d-d97abaac3342 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:17.957776 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6556a-b8b8-4312-b262-687191d58dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:17.958455 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6556a-b8b8-4312-b262-687191d58dff None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:17.958738 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6556a-b8b8-4312-b262-687191d58dff None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:17.959010 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6556a-b8b8-4312-b262-687191d58dff None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:17.961249 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6556a-b8b8-4312-b262-687191d58dff None admin] RBAC: Adding query filter params (name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:17.961249 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6556a-b8b8-4312-b262-687191d58dff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:17.961792 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6556a-b8b8-4312-b262-687191d58dff None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:17.972073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 608/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Mon May 9 09:37:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:17.977795 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0d74c-8022-4de3-8516-3d86c17a9f69 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:18.082946 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0d74c-8022-4de3-8516-3d86c17a9f69 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:18.117919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c874658-d09e-4ab7-b085-529d6bbe1671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:18.118715 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c874658-d09e-4ab7-b085-529d6bbe1671 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:18.119005 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c874658-d09e-4ab7-b085-529d6bbe1671 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:18.119270 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c874658-d09e-4ab7-b085-529d6bbe1671 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:18.130331 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c874658-d09e-4ab7-b085-529d6bbe1671 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:18.131817 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c874658-d09e-4ab7-b085-529d6bbe1671 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:18.150701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-2c874658-d09e-4ab7-b085-529d6bbe1671 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:18.150701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:18.150701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:18.150701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:18.150701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:18.150701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:18.150701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:18.150701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:18.150701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:18.150701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:18.150701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:18.150701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:18.150701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:37:18.150701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:37:18.150701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:37:18.152286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:37:18.152286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:18.152286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:18.152286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:18.152286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:18.152286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:18.152286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:18.152286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:18.152286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:37:18.152286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:18.152286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:18.152286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:18.152286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:18.152286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:18.152286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:18.152286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:18.152286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:18.153587 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:37:18.153587 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:37:18.153587 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:18.153587 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:37:18.153587 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 629/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:37:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:18.158019 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6556a-b8b8-4312-b262-687191d58dff None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:18.261605 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6556a-b8b8-4312-b262-687191d58dff None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:18.299858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f35eac-e138-4953-934c-9bee62e74732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:18.300552 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f35eac-e138-4953-934c-9bee62e74732 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:18.300853 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f35eac-e138-4953-934c-9bee62e74732 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:18.301127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f35eac-e138-4953-934c-9bee62e74732 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:18.301857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f35eac-e138-4953-934c-9bee62e74732 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:18.302170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f35eac-e138-4953-934c-9bee62e74732 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:18.303630 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f35eac-e138-4953-934c-9bee62e74732 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:18.315272 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 609/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:37:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:18.321424 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c874658-d09e-4ab7-b085-529d6bbe1671 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:18.428242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c874658-d09e-4ab7-b085-529d6bbe1671 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:18.467256 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830438e-b5ff-48a1-a199-b75a80aaaba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:18.468157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830438e-b5ff-48a1-a199-b75a80aaaba9 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:18.468506 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830438e-b5ff-48a1-a199-b75a80aaaba9 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:18.468893 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830438e-b5ff-48a1-a199-b75a80aaaba9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:18.477111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830438e-b5ff-48a1-a199-b75a80aaaba9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:18.478662 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830438e-b5ff-48a1-a199-b75a80aaaba9 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:18.494998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-9830438e-b5ff-48a1-a199-b75a80aaaba9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 09:37:18.494998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:18.494998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:18.494998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:18.494998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:18.494998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:18.494998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:18.494998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:18.494998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:18.494998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:18.494998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:18.494998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:18.494998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:37:18.494998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:37:18.494998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:37:18.494998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:37:18.496718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:18.496718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:18.496718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:37:18.496718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:37:18.496718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:37:18.496718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:37:18.496718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:18.496718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:18.496718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:18.496718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:18.496718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:18.496718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:37:18.496718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:18.496718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:18.496718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:18.496718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:18.496718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:18.496718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:18.499002 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:18.499002 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:18.499002 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:37:18.499002 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:37:18.499002 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:37:18.499002 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:37:18.499002 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:37:18.499002 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:37:18.499002 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 09:37:18.499002 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:37:18.499002 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 630/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:37:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 09:37:18.506506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f35eac-e138-4953-934c-9bee62e74732 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:18.617271 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f35eac-e138-4953-934c-9bee62e74732 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:18.655449 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16ce76-8e4e-4e50-b60c-a814dcdeff3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:18.656147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16ce76-8e4e-4e50-b60c-a814dcdeff3d None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:18.656406 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16ce76-8e4e-4e50-b60c-a814dcdeff3d None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:18.656693 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16ce76-8e4e-4e50-b60c-a814dcdeff3d None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:18.657623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16ce76-8e4e-4e50-b60c-a814dcdeff3d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:18.658005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16ce76-8e4e-4e50-b60c-a814dcdeff3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:18.659451 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16ce76-8e4e-4e50-b60c-a814dcdeff3d None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:18.680273 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 610/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Mon May 9 09:37:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:18.685875 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830438e-b5ff-48a1-a199-b75a80aaaba9 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:18.793007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830438e-b5ff-48a1-a199-b75a80aaaba9 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:18.828309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9927c7-c6ef-4c2b-8d66-c751b3317cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:18.829040 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9927c7-c6ef-4c2b-8d66-c751b3317cae None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:18.829320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9927c7-c6ef-4c2b-8d66-c751b3317cae None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:18.829598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9927c7-c6ef-4c2b-8d66-c751b3317cae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:18.899119 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9927c7-c6ef-4c2b-8d66-c751b3317cae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:18.900688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9927c7-c6ef-4c2b-8d66-c751b3317cae None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:19.001031 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 631/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:37:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 09:37:19.006866 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16ce76-8e4e-4e50-b60c-a814dcdeff3d None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:19.113922 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16ce76-8e4e-4e50-b60c-a814dcdeff3d None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:19.149036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b8709-ab7c-4983-9f17-f75c649e7b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:19.149835 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b8709-ab7c-4983-9f17-f75c649e7b83 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:19.150108 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b8709-ab7c-4983-9f17-f75c649e7b83 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:19.150381 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b8709-ab7c-4983-9f17-f75c649e7b83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:19.160616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b8709-ab7c-4983-9f17-f75c649e7b83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:19.162102 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b8709-ab7c-4983-9f17-f75c649e7b83 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:19.180655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-411b8709-ab7c-4983-9f17-f75c649e7b83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:19.180655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:19.180655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:19.180655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:19.180655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:19.180655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:19.180655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:19.180655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:19.180655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:19.180655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:19.180655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:19.180655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:19.180655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:37:19.180655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:37:19.180655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:37:19.181659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:37:19.181659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:19.181659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:19.181659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:19.181659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:19.181659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:19.181659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:19.181659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:19.181659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:37:19.181659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:19.181659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:19.181659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:19.181659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:19.181659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:19.181659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:19.181659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:19.181659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:19.182581 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:37:19.182581 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:37:19.182581 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:19.182581 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:19.182581 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 611/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:37:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:19.188429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9927c7-c6ef-4c2b-8d66-c751b3317cae None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:19.288826 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9927c7-c6ef-4c2b-8d66-c751b3317cae None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:19.322511 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046770f-8fcc-4a43-8346-8987e1b8a784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:19.323221 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046770f-8fcc-4a43-8346-8987e1b8a784 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:19.323524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046770f-8fcc-4a43-8346-8987e1b8a784 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:19.323817 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046770f-8fcc-4a43-8346-8987e1b8a784 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:19.324632 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7046770f-8fcc-4a43-8346-8987e1b8a784 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:19.324978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7046770f-8fcc-4a43-8346-8987e1b8a784 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:19.326348 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7046770f-8fcc-4a43-8346-8987e1b8a784 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:19.338603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 632/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:37:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:19.344678 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b8709-ab7c-4983-9f17-f75c649e7b83 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:19.446866 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b8709-ab7c-4983-9f17-f75c649e7b83 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:19.479542 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117931f-2e1a-44de-863c-f87a2cb2c0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:19.480326 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117931f-2e1a-44de-863c-f87a2cb2c0a2 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:19.480598 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117931f-2e1a-44de-863c-f87a2cb2c0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:19.480887 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117931f-2e1a-44de-863c-f87a2cb2c0a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:19.487530 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a117931f-2e1a-44de-863c-f87a2cb2c0a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:19.489036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a117931f-2e1a-44de-863c-f87a2cb2c0a2 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:19.504236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-a117931f-2e1a-44de-863c-f87a2cb2c0a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:37:19.504236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:19.504236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:19.504236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:19.504236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:19.504236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:19.504236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:19.504236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:19.504236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:19.504236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:19.504236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:19.504236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:19.504236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:37:19.504236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:37:19.504236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:37:19.505339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:37:19.505339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:19.505339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:19.505339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:19.505339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:19.505339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:19.505339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:19.505339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:19.505339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:37:19.505339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:19.505339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:19.505339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:19.505339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:19.505339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:19.505339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:19.505339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:19.505339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:19.506255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:37:19.506255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:37:19.506255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:37:19.506255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:37:19.506255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:37:19.506255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:37:19.506255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:37:19.506255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:19.506255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 612/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:37:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:19.512356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046770f-8fcc-4a43-8346-8987e1b8a784 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:19.619621 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046770f-8fcc-4a43-8346-8987e1b8a784 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:19.653136 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d13ff-0673-4efe-ab90-a32c20456726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:19.653772 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d13ff-0673-4efe-ab90-a32c20456726 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:19.654045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d13ff-0673-4efe-ab90-a32c20456726 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:19.654310 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d13ff-0673-4efe-ab90-a32c20456726 None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:19.655118 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d13ff-0673-4efe-ab90-a32c20456726 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:19.655423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d13ff-0673-4efe-ab90-a32c20456726 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:19.656961 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d13ff-0673-4efe-ab90-a32c20456726 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:19.668603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 633/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:37:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:19.674994 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117931f-2e1a-44de-863c-f87a2cb2c0a2 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:19.788354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117931f-2e1a-44de-863c-f87a2cb2c0a2 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:19.825433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea725f-cff4-407a-850c-ab248dc7507f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:19.826016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea725f-cff4-407a-850c-ab248dc7507f None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:19.826300 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea725f-cff4-407a-850c-ab248dc7507f None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:19.826653 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea725f-cff4-407a-850c-ab248dc7507f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:19.827580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea725f-cff4-407a-850c-ab248dc7507f None admin] RBAC: Adding query filter params (role.id=f55c3728df78466989d0cd1b13c5ea28, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=6c9e458c2c5f4ed1b3eddd84c2ec1703) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:19.827888 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea725f-cff4-407a-850c-ab248dc7507f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:19.829496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea725f-cff4-407a-850c-ab248dc7507f None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:19.835404 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 613/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:37:19 2022] GET /identity/v3/role_assignments?user.id=6c9e458c2c5f4ed1b3eddd84c2ec1703&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=f55c3728df78466989d0cd1b13c5ea28 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:22.072801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfaf49-d371-412c-b125-82a3014214dc None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:22.073566 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfaf49-d371-412c-b125-82a3014214dc None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:22.073566 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfaf49-d371-412c-b125-82a3014214dc None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:22.074809 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 634/1247] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:37:22.080736 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8407789-8936-4bf8-97b5-4a4e30ca4c1b None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:22.081209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8407789-8936-4bf8-97b5-4a4e30ca4c1b None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:22.081317 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8407789-8936-4bf8-97b5-4a4e30ca4c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:22.099100 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-b8407789-8936-4bf8-97b5-4a4e30ca4c1b None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:37:22.100418 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-b8407789-8936-4bf8-97b5-4a4e30ca4c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:37:22.167438 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 614/1248] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:37:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:37:22.192009 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfaf49-d371-412c-b125-82a3014214dc None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:22.299585 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfaf49-d371-412c-b125-82a3014214dc None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:22.335246 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd236d0-80cc-402f-97da-921ae76e8256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:22.336119 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd236d0-80cc-402f-97da-921ae76e8256 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:22.336400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd236d0-80cc-402f-97da-921ae76e8256 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:22.336696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd236d0-80cc-402f-97da-921ae76e8256 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:22.347654 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd236d0-80cc-402f-97da-921ae76e8256 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:37:22.356007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-ffd236d0-80cc-402f-97da-921ae76e8256 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:37:22.356007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:22.356007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:22.356007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:22.356007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:22.356007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:22.356007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:22.356007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:22.356007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:22.356007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:22.356007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:22.356007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:22.356007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:37:22.356007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:37:22.356007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:37:22.356007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:37:22.357309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:37:22.357309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:37:22.357309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:37:22.357309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:37:22.357309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:22.357309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:22.357309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:22.357309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:22.357309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:22.357309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:22.357309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:22.357309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:37:22.357309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:22.357309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:22.357309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:22.357309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:22.358248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:22.358248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:22.358248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:22.358248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:22.358248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:37:22.358248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:37:22.358248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:37:22.358248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:37:22.358248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:37:22.358248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:37:22.358248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:37:22.358248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:37:22.358248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 635/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:37:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:22.363968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8407789-8936-4bf8-97b5-4a4e30ca4c1b None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:22.467277 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8407789-8936-4bf8-97b5-4a4e30ca4c1b None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:22.503557 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c15710-21ce-4eb2-90c5-c2cedaf3e387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:22.504795 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c15710-21ce-4eb2-90c5-c2cedaf3e387 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:22.504795 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c15710-21ce-4eb2-90c5-c2cedaf3e387 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:22.504795 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c15710-21ce-4eb2-90c5-c2cedaf3e387 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:22.505500 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c15710-21ce-4eb2-90c5-c2cedaf3e387 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:22.505804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c15710-21ce-4eb2-90c5-c2cedaf3e387 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:22.507167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c15710-21ce-4eb2-90c5-c2cedaf3e387 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:22.515216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 615/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:37:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:22.521215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd236d0-80cc-402f-97da-921ae76e8256 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:22.620949 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd236d0-80cc-402f-97da-921ae76e8256 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:22.653377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b90cf-5398-4371-b0bf-128ddb3729ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:22.654335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b90cf-5398-4371-b0bf-128ddb3729ed None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:22.654711 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b90cf-5398-4371-b0bf-128ddb3729ed None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:22.655039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b90cf-5398-4371-b0bf-128ddb3729ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:22.665739 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b90cf-5398-4371-b0bf-128ddb3729ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:22.667214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b90cf-5398-4371-b0bf-128ddb3729ed None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:22.684697 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-c15b90cf-5398-4371-b0bf-128ddb3729ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:22.684697 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:22.684697 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:22.684697 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:22.684697 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:22.684697 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:22.684697 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:22.684697 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:22.684697 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:22.684697 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:22.684697 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:22.684697 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:22.684697 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:37:22.684697 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:37:22.684697 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:37:22.687158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:37:22.687158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:22.687158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:22.687158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:22.687158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:22.687158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:22.687158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:22.687158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:22.687158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:37:22.687158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:22.687158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:22.687158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:22.687158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:22.687158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:22.687158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:22.687158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:22.687158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:22.688525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:37:22.688525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:37:22.688525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:22.688525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:37:22.688525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 636/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:37:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:22.691972 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c15710-21ce-4eb2-90c5-c2cedaf3e387 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:22.806300 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c15710-21ce-4eb2-90c5-c2cedaf3e387 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:22.840977 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee768ce-2440-4abb-9771-efe45b935804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:22.841642 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee768ce-2440-4abb-9771-efe45b935804 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:22.841862 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee768ce-2440-4abb-9771-efe45b935804 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:22.842134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee768ce-2440-4abb-9771-efe45b935804 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:22.842846 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee768ce-2440-4abb-9771-efe45b935804 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:22.843144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee768ce-2440-4abb-9771-efe45b935804 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:22.844845 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee768ce-2440-4abb-9771-efe45b935804 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:22.856961 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 616/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:22.862557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b90cf-5398-4371-b0bf-128ddb3729ed None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:22.961192 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b90cf-5398-4371-b0bf-128ddb3729ed None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:22.996378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0278b0e-adfa-4792-a1ca-df1fad1b27d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:22.997281 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0278b0e-adfa-4792-a1ca-df1fad1b27d7 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:22.997555 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0278b0e-adfa-4792-a1ca-df1fad1b27d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:22.997821 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0278b0e-adfa-4792-a1ca-df1fad1b27d7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:23.006843 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0278b0e-adfa-4792-a1ca-df1fad1b27d7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:23.008288 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0278b0e-adfa-4792-a1ca-df1fad1b27d7 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:23.025579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-d0278b0e-adfa-4792-a1ca-df1fad1b27d7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 09:37:23.025579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:23.025579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:23.025579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:23.025579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:23.025579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:23.025579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:23.025579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:23.025579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:23.025579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:23.025579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:23.025579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:23.025579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:37:23.025579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:37:23.025579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:37:23.025579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:37:23.026903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:23.026903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:23.026903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:37:23.026903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:37:23.026903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:37:23.026903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:37:23.026903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:23.026903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:23.026903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:23.026903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:23.026903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:23.026903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:37:23.026903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:23.026903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:23.026903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:23.026903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:23.026903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:23.026903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:23.028157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:23.028157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:23.028157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:37:23.028157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:37:23.028157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:37:23.028157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:37:23.028157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:37:23.028157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:37:23.028157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 09:37:23.028157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:37:23.028157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 637/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:37:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:23.032908 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee768ce-2440-4abb-9771-efe45b935804 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:23.138892 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee768ce-2440-4abb-9771-efe45b935804 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:23.174232 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ab117-789d-47e4-8555-dca128b91218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:23.174890 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ab117-789d-47e4-8555-dca128b91218 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:23.175162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ab117-789d-47e4-8555-dca128b91218 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:23.175426 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ab117-789d-47e4-8555-dca128b91218 None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:23.176258 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ab117-789d-47e4-8555-dca128b91218 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:23.176555 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ab117-789d-47e4-8555-dca128b91218 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:23.178040 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ab117-789d-47e4-8555-dca128b91218 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:23.200043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 617/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Mon May 9 09:37:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:23.201410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0278b0e-adfa-4792-a1ca-df1fad1b27d7 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:23.306733 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0278b0e-adfa-4792-a1ca-df1fad1b27d7 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:23.343628 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9127d-749e-4740-8431-3f40b2f5d099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:23.344489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9127d-749e-4740-8431-3f40b2f5d099 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:23.344801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9127d-749e-4740-8431-3f40b2f5d099 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:23.345069 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9127d-749e-4740-8431-3f40b2f5d099 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:23.425818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb9127d-749e-4740-8431-3f40b2f5d099 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:23.427470 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb9127d-749e-4740-8431-3f40b2f5d099 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:23.527793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 638/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:37:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 09:37:23.534399 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ab117-789d-47e4-8555-dca128b91218 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:23.635874 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ab117-789d-47e4-8555-dca128b91218 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:23.672306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e605e-45de-47ea-a676-c03669c4d677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:23.673293 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e605e-45de-47ea-a676-c03669c4d677 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:23.673631 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e605e-45de-47ea-a676-c03669c4d677 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:23.674000 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e605e-45de-47ea-a676-c03669c4d677 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:23.684104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e605e-45de-47ea-a676-c03669c4d677 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:23.685597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e605e-45de-47ea-a676-c03669c4d677 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:23.704073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-642e605e-45de-47ea-a676-c03669c4d677 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:23.704073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:23.704073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:23.704073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:23.704073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:23.704073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:23.704073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:23.704073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:23.704073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:23.704073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:23.704073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:23.704073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:23.704073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:37:23.704073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:37:23.704073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:37:23.705491 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:37:23.705491 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:23.705491 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:23.705491 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:23.705491 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:23.705491 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:23.705491 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:23.705491 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:23.705491 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:37:23.705491 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:23.705491 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:23.705491 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:23.705491 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:23.705491 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:23.705491 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:23.705491 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:23.705491 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:23.706422 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:37:23.706422 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:37:23.706422 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:23.706422 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:23.706422 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 618/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:37:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:23.712722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9127d-749e-4740-8431-3f40b2f5d099 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:23.827929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9127d-749e-4740-8431-3f40b2f5d099 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:23.863591 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55121c12-7094-4fd4-975a-3c6993002c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:23.864312 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55121c12-7094-4fd4-975a-3c6993002c1f None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:23.864641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55121c12-7094-4fd4-975a-3c6993002c1f None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:23.864883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55121c12-7094-4fd4-975a-3c6993002c1f None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:23.865916 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55121c12-7094-4fd4-975a-3c6993002c1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:23.866217 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55121c12-7094-4fd4-975a-3c6993002c1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:23.867596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55121c12-7094-4fd4-975a-3c6993002c1f None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:23.879024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 639/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:37:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:23.884982 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e605e-45de-47ea-a676-c03669c4d677 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:23.991905 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e605e-45de-47ea-a676-c03669c4d677 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:24.029082 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a8550-e8bb-4449-810b-99cede5b2ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:24.029896 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a8550-e8bb-4449-810b-99cede5b2ed2 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:24.030172 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a8550-e8bb-4449-810b-99cede5b2ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:24.030443 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a8550-e8bb-4449-810b-99cede5b2ed2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:24.038468 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a8550-e8bb-4449-810b-99cede5b2ed2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:24.039957 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a8550-e8bb-4449-810b-99cede5b2ed2 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:24.054924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-997a8550-e8bb-4449-810b-99cede5b2ed2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:37:24.054924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:24.054924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:24.054924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:24.054924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:24.054924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:24.054924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:24.054924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:24.054924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:24.054924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:24.054924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:24.054924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:24.054924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:37:24.054924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:37:24.054924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:37:24.056075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:37:24.056075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:24.056075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:24.056075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:24.056075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:24.056075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:24.056075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:24.056075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:24.056075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:37:24.056075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:24.056075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:24.056075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:24.056075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:24.056075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:24.056075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:24.056075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:24.056075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:24.057160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:37:24.057160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:37:24.057160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:37:24.057160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:37:24.057160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:37:24.057160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:37:24.057160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:37:24.057160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:24.057160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 619/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:37:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:24.062305 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55121c12-7094-4fd4-975a-3c6993002c1f None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:24.165578 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55121c12-7094-4fd4-975a-3c6993002c1f None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:24.201128 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c2bed-763e-4c59-9895-07626d1cb9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:24.201798 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c2bed-763e-4c59-9895-07626d1cb9a8 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:24.202414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c2bed-763e-4c59-9895-07626d1cb9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:24.202414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c2bed-763e-4c59-9895-07626d1cb9a8 None admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:24.203116 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c2bed-763e-4c59-9895-07626d1cb9a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:24.203464 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c2bed-763e-4c59-9895-07626d1cb9a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:24.204957 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c2bed-763e-4c59-9895-07626d1cb9a8 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:24.218139 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 640/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:37:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:24.224605 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a8550-e8bb-4449-810b-99cede5b2ed2 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:24.330949 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a8550-e8bb-4449-810b-99cede5b2ed2 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:24.367504 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7ad0c-b839-4741-8756-98744c9e86c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:24.368161 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7ad0c-b839-4741-8756-98744c9e86c0 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:24.368437 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7ad0c-b839-4741-8756-98744c9e86c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:24.368706 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7ad0c-b839-4741-8756-98744c9e86c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:24.369576 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7ad0c-b839-4741-8756-98744c9e86c0 None admin] RBAC: Adding query filter params (role.id=70ac549382b14b2cbb8b35e5dfc648d4, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=6c9e458c2c5f4ed1b3eddd84c2ec1703) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:24.369882 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7ad0c-b839-4741-8756-98744c9e86c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:24.371323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7ad0c-b839-4741-8756-98744c9e86c0 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:24.377523 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 620/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:37:24 2022] GET /identity/v3/role_assignments?user.id=6c9e458c2c5f4ed1b3eddd84c2ec1703&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=70ac549382b14b2cbb8b35e5dfc648d4 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:26.547899 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00636709-7e27-423c-a41e-f4181d1f4230 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:26.548890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00636709-7e27-423c-a41e-f4181d1f4230 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:26.549049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00636709-7e27-423c-a41e-f4181d1f4230 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:26.551204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 641/1261] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:37:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:37:26.557585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276dc45d-d0b1-4fee-a29d-0573cfaf47cf None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:26.558048 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276dc45d-d0b1-4fee-a29d-0573cfaf47cf None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:26.558159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276dc45d-d0b1-4fee-a29d-0573cfaf47cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:26.577471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-276dc45d-d0b1-4fee-a29d-0573cfaf47cf None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:37:26.578643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-276dc45d-d0b1-4fee-a29d-0573cfaf47cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:37:26.659964 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 621/1262] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:37:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:37:26.683215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00636709-7e27-423c-a41e-f4181d1f4230 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:26.806526 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00636709-7e27-423c-a41e-f4181d1f4230 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:26.841012 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d5ea1-ed4a-4dcc-a386-db6a884bc05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:26.841792 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d5ea1-ed4a-4dcc-a386-db6a884bc05d None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:26.842069 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d5ea1-ed4a-4dcc-a386-db6a884bc05d None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:26.842333 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d5ea1-ed4a-4dcc-a386-db6a884bc05d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:26.852987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d5ea1-ed4a-4dcc-a386-db6a884bc05d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:37:26.860524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-f71d5ea1-ed4a-4dcc-a386-db6a884bc05d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:37:26.860524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:26.860524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:26.860524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:26.860524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:26.860524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:26.860524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:26.860524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:26.860524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:26.860524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:26.860524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:26.860524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:26.860524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:37:26.860524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:37:26.860524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:37:26.860524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:37:26.861598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:37:26.861598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:37:26.861598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:37:26.861598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:37:26.861598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:26.861598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:26.861598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:26.861598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:26.861598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:26.861598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:26.861598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:26.861598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:37:26.861598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:26.861598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:26.861598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:26.861598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:26.862604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:26.862604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:26.862604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:26.862604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:26.862604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:37:26.862604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:37:26.862604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:37:26.862604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:37:26.862604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:37:26.862604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:37:26.862604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:37:26.862604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:37:26.862604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 642/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:37:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:26.871340 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276dc45d-d0b1-4fee-a29d-0573cfaf47cf None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:26.980152 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276dc45d-d0b1-4fee-a29d-0573cfaf47cf None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:27.014617 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1bca1-7e9b-4e6f-b815-91f3a0702416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:27.015279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1bca1-7e9b-4e6f-b815-91f3a0702416 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:27.015554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1bca1-7e9b-4e6f-b815-91f3a0702416 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:27.015837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1bca1-7e9b-4e6f-b815-91f3a0702416 None admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:27.016613 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1bca1-7e9b-4e6f-b815-91f3a0702416 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:27.016955 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1bca1-7e9b-4e6f-b815-91f3a0702416 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:27.018331 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1bca1-7e9b-4e6f-b815-91f3a0702416 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:27.026311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 622/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:37:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:27.032945 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d5ea1-ed4a-4dcc-a386-db6a884bc05d None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:27.148074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d5ea1-ed4a-4dcc-a386-db6a884bc05d None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:27.193883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b72a2-572c-467f-bc01-d683b06bb767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:27.194651 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b72a2-572c-467f-bc01-d683b06bb767 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:27.194920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b72a2-572c-467f-bc01-d683b06bb767 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:27.195180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b72a2-572c-467f-bc01-d683b06bb767 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:27.206413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b72a2-572c-467f-bc01-d683b06bb767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:27.207852 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b72a2-572c-467f-bc01-d683b06bb767 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:27.225301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-e12b72a2-572c-467f-bc01-d683b06bb767 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:27.225301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:27.225301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:27.225301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:27.225301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:27.225301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:27.225301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:27.225301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:27.225301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:27.225301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:27.225301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:27.225301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:27.225301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:37:27.225301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:37:27.225301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:37:27.226342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:37:27.226342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:27.226342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:27.226342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:27.226342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:27.226342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:27.226342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:27.226342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:27.226342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:37:27.226342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:27.226342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:27.226342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:27.226342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:27.226342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:27.226342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:27.226342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:27.226342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:27.227383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:37:27.227383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:37:27.227383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:27.227383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:37:27.227383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 643/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:37:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:27.233047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1bca1-7e9b-4e6f-b815-91f3a0702416 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:27.337804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1bca1-7e9b-4e6f-b815-91f3a0702416 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:27.371717 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c0286-e056-4abc-9d97-92f61d5784c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:27.372355 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c0286-e056-4abc-9d97-92f61d5784c9 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:27.372632 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c0286-e056-4abc-9d97-92f61d5784c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:27.372925 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c0286-e056-4abc-9d97-92f61d5784c9 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:27.373646 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c0286-e056-4abc-9d97-92f61d5784c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:27.373954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c0286-e056-4abc-9d97-92f61d5784c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:27.375391 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c0286-e056-4abc-9d97-92f61d5784c9 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:27.386888 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 623/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:27.395689 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b72a2-572c-467f-bc01-d683b06bb767 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:27.503574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b72a2-572c-467f-bc01-d683b06bb767 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:27.536500 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b29bc-a144-4a45-be6a-ca7d39360d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:27.537303 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b29bc-a144-4a45-be6a-ca7d39360d19 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:27.537613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b29bc-a144-4a45-be6a-ca7d39360d19 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:27.537854 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b29bc-a144-4a45-be6a-ca7d39360d19 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:27.545571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b29bc-a144-4a45-be6a-ca7d39360d19 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:27.547063 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b29bc-a144-4a45-be6a-ca7d39360d19 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:27.561870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-e58b29bc-a144-4a45-be6a-ca7d39360d19 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 09:37:27.561870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:27.561870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:27.561870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:27.561870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:27.561870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:27.561870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:27.561870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:27.561870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:27.561870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:27.561870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:27.561870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:27.561870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:37:27.561870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:37:27.561870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:37:27.561870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:37:27.563483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:27.563483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:27.563483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:37:27.563483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:37:27.563483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:37:27.563483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:37:27.563483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:27.563483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:27.563483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:27.563483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:27.563483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:27.563483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:37:27.563483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:27.563483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:27.563483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:27.563483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:27.563483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:27.563483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:27.564569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:27.564569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:27.564569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:37:27.564569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:37:27.564569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:37:27.564569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:37:27.564569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:37:27.564569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:37:27.564569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 09:37:27.564569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:37:27.564569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 644/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:37:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:27.569181 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c0286-e056-4abc-9d97-92f61d5784c9 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:27.666700 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c0286-e056-4abc-9d97-92f61d5784c9 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:27.700350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303a7cf-73fe-4fb9-a022-ca4bb7be6fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:27.701027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303a7cf-73fe-4fb9-a022-ca4bb7be6fe7 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:27.701301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303a7cf-73fe-4fb9-a022-ca4bb7be6fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:27.701585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303a7cf-73fe-4fb9-a022-ca4bb7be6fe7 None admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:27.702399 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4303a7cf-73fe-4fb9-a022-ca4bb7be6fe7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:27.702704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4303a7cf-73fe-4fb9-a022-ca4bb7be6fe7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:27.704178 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4303a7cf-73fe-4fb9-a022-ca4bb7be6fe7 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:27.720790 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 624/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Mon May 9 09:37:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:27.727139 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b29bc-a144-4a45-be6a-ca7d39360d19 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:27.848050 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b29bc-a144-4a45-be6a-ca7d39360d19 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:27.883694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6634f0-487e-4bfe-9bc7-6177f85c174d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:27.884476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6634f0-487e-4bfe-9bc7-6177f85c174d None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:27.884776 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6634f0-487e-4bfe-9bc7-6177f85c174d None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:27.885045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6634f0-487e-4bfe-9bc7-6177f85c174d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:27.895654 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6634f0-487e-4bfe-9bc7-6177f85c174d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:27.897171 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6634f0-487e-4bfe-9bc7-6177f85c174d None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:27.914062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-bc6634f0-487e-4bfe-9bc7-6177f85c174d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:27.914062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:27.914062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:27.914062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:27.914062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:27.914062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:27.914062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:27.914062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:27.914062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:27.914062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:27.914062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:27.914062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:27.914062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:37:27.914062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:37:27.914062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:37:27.915074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:37:27.915074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:27.915074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:27.915074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:27.915074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:27.915074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:27.915074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:27.915074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:27.915074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:37:27.915074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:27.915074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:27.915074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:27.915074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:27.915074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:27.915074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:27.915074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:27.915074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:27.916470 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:37:27.916470 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:37:27.916470 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:27.916470 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:37:27.916470 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 645/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:37:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:27.924114 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303a7cf-73fe-4fb9-a022-ca4bb7be6fe7 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:28.030748 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303a7cf-73fe-4fb9-a022-ca4bb7be6fe7 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:28.064320 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609c70e-4350-499a-98b0-32e94695eef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:28.064987 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609c70e-4350-499a-98b0-32e94695eef3 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:28.065258 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609c70e-4350-499a-98b0-32e94695eef3 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:28.065517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609c70e-4350-499a-98b0-32e94695eef3 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:28.066230 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609c70e-4350-499a-98b0-32e94695eef3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:28.066527 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609c70e-4350-499a-98b0-32e94695eef3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:28.067876 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609c70e-4350-499a-98b0-32e94695eef3 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:28.078972 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 625/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:28.084686 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6634f0-487e-4bfe-9bc7-6177f85c174d None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:28.185359 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6634f0-487e-4bfe-9bc7-6177f85c174d None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:28.218213 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa0a10-0023-43c1-9eff-8146e1df0668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:28.219027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa0a10-0023-43c1-9eff-8146e1df0668 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:28.219301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa0a10-0023-43c1-9eff-8146e1df0668 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:28.219567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa0a10-0023-43c1-9eff-8146e1df0668 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:28.226371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa0a10-0023-43c1-9eff-8146e1df0668 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:28.227801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa0a10-0023-43c1-9eff-8146e1df0668 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:28.242886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-c6fa0a10-0023-43c1-9eff-8146e1df0668 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:37:28.242886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:28.242886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:28.242886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:28.242886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:28.242886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:28.242886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:28.242886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:28.242886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:28.242886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:28.242886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:28.242886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:28.242886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:37:28.242886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:37:28.242886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:37:28.243936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:37:28.243936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:28.243936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:28.243936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:28.243936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:28.243936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:28.243936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:28.243936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:28.243936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:37:28.243936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:28.243936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:28.243936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:28.243936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:28.243936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:28.243936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:28.243936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:28.243936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:28.245325 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:37:28.245325 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:37:28.245325 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:37:28.245325 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:37:28.245325 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:37:28.245325 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:37:28.245325 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:37:28.245325 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:37:28.245325 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 646/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:37:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:28.250431 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609c70e-4350-499a-98b0-32e94695eef3 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:28.356382 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609c70e-4350-499a-98b0-32e94695eef3 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:28.390074 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3c987-7fd9-461c-b4d6-d52a2566b5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:28.390763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3c987-7fd9-461c-b4d6-d52a2566b5d6 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:28.391040 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3c987-7fd9-461c-b4d6-d52a2566b5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:28.391306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3c987-7fd9-461c-b4d6-d52a2566b5d6 None admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:28.392093 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e3c987-7fd9-461c-b4d6-d52a2566b5d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:28.392393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e3c987-7fd9-461c-b4d6-d52a2566b5d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:28.393856 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e3c987-7fd9-461c-b4d6-d52a2566b5d6 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:28.405646 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 626/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:37:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:28.414869 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa0a10-0023-43c1-9eff-8146e1df0668 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:28.519914 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa0a10-0023-43c1-9eff-8146e1df0668 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:28.567544 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed9ac4-0be1-4370-9fed-d8769f09ec62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:28.568430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed9ac4-0be1-4370-9fed-d8769f09ec62 None admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:28.568699 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed9ac4-0be1-4370-9fed-d8769f09ec62 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:28.569000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed9ac4-0be1-4370-9fed-d8769f09ec62 None admin] PATH_INFO: `/v3/projects/9cd39464055141b8b43fde20d0885c99/users/6c9e458c2c5f4ed1b3eddd84c2ec1703/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:28.571389 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ed9ac4-0be1-4370-9fed-d8769f09ec62 None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd39464055141b8b43fde20d0885c99, user_id=6c9e458c2c5f4ed1b3eddd84c2ec1703, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:28.572872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ed9ac4-0be1-4370-9fed-d8769f09ec62 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:28.589497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 647/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Mon May 9 09:37:28 2022] PUT /identity/v3/projects/9cd39464055141b8b43fde20d0885c99/users/6c9e458c2c5f4ed1b3eddd84c2ec1703/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:37:30.782916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd2cef-8733-4648-a4b8-a1bd02412ec3 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:30.783835 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd2cef-8733-4648-a4b8-a1bd02412ec3 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:30.783835 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd2cef-8733-4648-a4b8-a1bd02412ec3 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:30.786000 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 627/1274] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:37:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:37:30.792316 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a4747-496d-4b29-a411-57c6b7d4da54 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:30.793053 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a4747-496d-4b29-a411-57c6b7d4da54 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:30.793255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a4747-496d-4b29-a411-57c6b7d4da54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:30.818074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-413a4747-496d-4b29-a411-57c6b7d4da54 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:37:30.819180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-413a4747-496d-4b29-a411-57c6b7d4da54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:37:30.886372 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 648/1275] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:37:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:37:30.917660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd2cef-8733-4648-a4b8-a1bd02412ec3 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:31.024742 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd2cef-8733-4648-a4b8-a1bd02412ec3 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:31.058553 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d035a-eff3-49e0-8ff3-b22eda11bf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:31.059323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d035a-eff3-49e0-8ff3-b22eda11bf1e None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:31.059602 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d035a-eff3-49e0-8ff3-b22eda11bf1e None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:31.059875 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d035a-eff3-49e0-8ff3-b22eda11bf1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:31.070490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d035a-eff3-49e0-8ff3-b22eda11bf1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 09:37:31.077954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-b01d035a-eff3-49e0-8ff3-b22eda11bf1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:37:31.077954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:31.077954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:31.077954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:31.077954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:31.077954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:31.077954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:31.077954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:31.077954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:31.077954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:31.077954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:31.077954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:31.077954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 09:37:31.077954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 09:37:31.077954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 09:37:31.077954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 09:37:31.079009 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 09:37:31.079009 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 09:37:31.079009 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 09:37:31.079009 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 09:37:31.079009 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:31.079009 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:31.079009 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:31.079009 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:31.079009 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:31.079009 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:31.079009 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:31.079009 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:37:31.079009 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:31.079009 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:31.079009 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:31.079009 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:31.079966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:31.079966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:31.079966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:31.079966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:31.079966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 09:37:31.079966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 09:37:31.079966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 09:37:31.079966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 09:37:31.079966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 09:37:31.079966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 09:37:31.079966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 09:37:31.079966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:31.079966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 628/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Mon May 9 09:37:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:31.085455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a4747-496d-4b29-a411-57c6b7d4da54 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:31.182164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a4747-496d-4b29-a411-57c6b7d4da54 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:31.215037 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3fbed-7fa7-4f35-8d7c-fc3bb9dd0a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:31.215676 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3fbed-7fa7-4f35-8d7c-fc3bb9dd0a20 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:31.215945 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3fbed-7fa7-4f35-8d7c-fc3bb9dd0a20 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:31.216207 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3fbed-7fa7-4f35-8d7c-fc3bb9dd0a20 None admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:31.216972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf3fbed-7fa7-4f35-8d7c-fc3bb9dd0a20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:31.217265 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf3fbed-7fa7-4f35-8d7c-fc3bb9dd0a20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:31.218650 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf3fbed-7fa7-4f35-8d7c-fc3bb9dd0a20 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:31.226411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 649/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Mon May 9 09:37:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:31.232720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d035a-eff3-49e0-8ff3-b22eda11bf1e None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:31.336955 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d035a-eff3-49e0-8ff3-b22eda11bf1e None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:31.370422 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372f00f-5aba-43bf-aef8-b08d69d124e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:31.371190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372f00f-5aba-43bf-aef8-b08d69d124e9 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:31.371465 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372f00f-5aba-43bf-aef8-b08d69d124e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:31.371731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372f00f-5aba-43bf-aef8-b08d69d124e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:31.382368 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a372f00f-5aba-43bf-aef8-b08d69d124e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:31.383805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a372f00f-5aba-43bf-aef8-b08d69d124e9 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:31.400723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-a372f00f-5aba-43bf-aef8-b08d69d124e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:31.400723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:31.400723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:31.400723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:31.400723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:31.400723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:31.400723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:31.400723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:31.400723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:31.400723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:31.400723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:31.400723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:31.400723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:37:31.400723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:37:31.400723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:37:31.401745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:37:31.401745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:31.401745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:31.401745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:31.401745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:31.401745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:31.401745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:31.401745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:31.401745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:37:31.401745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:31.401745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:31.401745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:31.401745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:31.401745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:31.401745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:31.401745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:31.401745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:31.403398 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:37:31.403398 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:37:31.403398 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:31.403398 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:31.403398 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 629/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:37:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:31.410611 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3fbed-7fa7-4f35-8d7c-fc3bb9dd0a20 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:31.515120 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3fbed-7fa7-4f35-8d7c-fc3bb9dd0a20 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:31.549305 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32234adb-2c64-443f-8825-60fd071025f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:31.549946 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32234adb-2c64-443f-8825-60fd071025f1 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:31.550219 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32234adb-2c64-443f-8825-60fd071025f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:31.550484 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32234adb-2c64-443f-8825-60fd071025f1 None admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:31.551191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32234adb-2c64-443f-8825-60fd071025f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:31.551494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32234adb-2c64-443f-8825-60fd071025f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:31.552891 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32234adb-2c64-443f-8825-60fd071025f1 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:31.566305 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 650/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:31.572044 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372f00f-5aba-43bf-aef8-b08d69d124e9 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:31.676545 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372f00f-5aba-43bf-aef8-b08d69d124e9 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:31.714725 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d8875-1c50-49f3-b33b-b15027b435ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:31.715864 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d8875-1c50-49f3-b33b-b15027b435ed None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:31.716176 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d8875-1c50-49f3-b33b-b15027b435ed None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:31.716482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d8875-1c50-49f3-b33b-b15027b435ed None admin] PATH_INFO: `/v3/users/placement` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:31.725551 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d8875-1c50-49f3-b33b-b15027b435ed None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:31.727270 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d8875-1c50-49f3-b33b-b15027b435ed None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:31.743939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-801d8875-1c50-49f3-b33b-b15027b435ed None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 09:37:31.743939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:31.743939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:31.743939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:31.743939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:31.743939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:31.743939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:31.743939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:31.743939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:31.743939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:31.743939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:31.743939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:31.743939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 09:37:31.743939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 09:37:31.743939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 09:37:31.743939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 09:37:31.745265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:31.745265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:31.745265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 09:37:31.745265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:37:31.745265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 09:37:31.745265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 09:37:31.745265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:31.745265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:31.745265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:31.745265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:31.745265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:31.745265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:37:31.745265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:31.745265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:31.745265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:31.745265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:31.745265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:31.745265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:31.746709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:31.746709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:31.746709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 09:37:31.746709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 09:37:31.746709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 09:37:31.746709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 09:37:31.746709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 09:37:31.746709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 09:37:31.746709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 09:37:31.746709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:31.746709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 630/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:37:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:31.751384 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32234adb-2c64-443f-8825-60fd071025f1 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:31.855638 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32234adb-2c64-443f-8825-60fd071025f1 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:31.889862 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15c87f-7d96-4e13-bb7a-59a797b6f46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:31.890543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15c87f-7d96-4e13-bb7a-59a797b6f46c None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:31.890815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15c87f-7d96-4e13-bb7a-59a797b6f46c None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:31.891088 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15c87f-7d96-4e13-bb7a-59a797b6f46c None admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:31.891912 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15c87f-7d96-4e13-bb7a-59a797b6f46c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:31.892219 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15c87f-7d96-4e13-bb7a-59a797b6f46c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:31.893669 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15c87f-7d96-4e13-bb7a-59a797b6f46c None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:31.910972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 651/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Mon May 9 09:37:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:37:31.917031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d8875-1c50-49f3-b33b-b15027b435ed None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:32.020064 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d8875-1c50-49f3-b33b-b15027b435ed None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:32.057336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c410e5e-cdcc-43cd-8355-253833a387c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:32.057958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c410e5e-cdcc-43cd-8355-253833a387c1 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:32.058233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c410e5e-cdcc-43cd-8355-253833a387c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:32.058496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c410e5e-cdcc-43cd-8355-253833a387c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:32.135301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c410e5e-cdcc-43cd-8355-253833a387c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:32.136779 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c410e5e-cdcc-43cd-8355-253833a387c1 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:32.237572 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 631/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Mon May 9 09:37:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 09:37:32.247089 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15c87f-7d96-4e13-bb7a-59a797b6f46c None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:32.352825 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15c87f-7d96-4e13-bb7a-59a797b6f46c None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:32.386761 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d2370-2340-4c46-bdc1-4de8e3c29867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:32.387515 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d2370-2340-4c46-bdc1-4de8e3c29867 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:32.387790 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d2370-2340-4c46-bdc1-4de8e3c29867 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:32.388055 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d2370-2340-4c46-bdc1-4de8e3c29867 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:32.398166 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d2370-2340-4c46-bdc1-4de8e3c29867 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:32.399598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d2370-2340-4c46-bdc1-4de8e3c29867 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:32.416755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-4a7d2370-2340-4c46-bdc1-4de8e3c29867 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:32.416755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:32.416755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:32.416755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:32.416755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:32.416755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:32.416755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:32.416755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:32.416755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:32.416755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:32.416755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:32.416755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:32.416755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 09:37:32.416755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 09:37:32.416755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 09:37:32.417764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 09:37:32.417764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:32.417764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:32.417764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:32.417764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:32.417764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:32.417764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:32.417764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:32.417764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:37:32.417764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:32.417764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:32.417764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:32.417764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:32.417764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:32.417764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:32.417764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:32.417764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:32.419036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 09:37:32.419036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 09:37:32.419036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 09:37:32.419036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:37:32.419036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 652/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:37:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:32.425493 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c410e5e-cdcc-43cd-8355-253833a387c1 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:32.521692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c410e5e-cdcc-43cd-8355-253833a387c1 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:32.557484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698602b8-a0e0-48cb-9ab3-c9d18e888084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:32.558139 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698602b8-a0e0-48cb-9ab3-c9d18e888084 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:32.558412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698602b8-a0e0-48cb-9ab3-c9d18e888084 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:32.558660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698602b8-a0e0-48cb-9ab3-c9d18e888084 None admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:32.559411 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698602b8-a0e0-48cb-9ab3-c9d18e888084 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:32.559666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698602b8-a0e0-48cb-9ab3-c9d18e888084 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:32.561241 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698602b8-a0e0-48cb-9ab3-c9d18e888084 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:32.573853 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 632/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Mon May 9 09:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:32.582215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d2370-2340-4c46-bdc1-4de8e3c29867 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:32.680756 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d2370-2340-4c46-bdc1-4de8e3c29867 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:32.717442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcd6de-0372-4a99-86ca-426920a02540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:32.718310 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcd6de-0372-4a99-86ca-426920a02540 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:32.718674 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcd6de-0372-4a99-86ca-426920a02540 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:32.718973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcd6de-0372-4a99-86ca-426920a02540 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:32.726803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcd6de-0372-4a99-86ca-426920a02540 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:32.728533 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcd6de-0372-4a99-86ca-426920a02540 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:32.746377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application [None req-8adcd6de-0372-4a99-86ca-426920a02540 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:37:32.746377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:32.746377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:32.746377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:32.746377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:32.746377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:32.746377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:32.746377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:32.746377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:32.746377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:32.746377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:32.746377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:32.746377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 09:37:32.746377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 09:37:32.746377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 09:37:32.747476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 09:37:32.747476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:32.747476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:32.747476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:32.747476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:32.747476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:32.747476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:32.747476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:32.747476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application with Lock( May 09 09:37:32.747476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:32.747476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:32.747476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:32.747476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:32.747476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:32.747476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:32.747476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:32.747476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:32.748618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 09:37:32.748618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 09:37:32.748618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 09:37:32.748618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 09:37:32.748618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 09:37:32.748618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 09:37:32.748618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 09:37:32.748618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: ERROR keystone.server.flask.application May 09 09:37:32.748618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 653/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Mon May 9 09:37:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:32.753681 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698602b8-a0e0-48cb-9ab3-c9d18e888084 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:32.850367 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698602b8-a0e0-48cb-9ab3-c9d18e888084 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:32.884183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda0fdb-bc60-418d-88c3-4cca5a136d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:32.884856 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda0fdb-bc60-418d-88c3-4cca5a136d69 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:32.885129 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda0fdb-bc60-418d-88c3-4cca5a136d69 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:32.885390 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda0fdb-bc60-418d-88c3-4cca5a136d69 None admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:32.886164 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda0fdb-bc60-418d-88c3-4cca5a136d69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:32.886464 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda0fdb-bc60-418d-88c3-4cca5a136d69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:32.887885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda0fdb-bc60-418d-88c3-4cca5a136d69 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:32.899691 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 633/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Mon May 9 09:37:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:32.906846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcd6de-0372-4a99-86ca-426920a02540 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:33.018182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcd6de-0372-4a99-86ca-426920a02540 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:33.052185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfaa98-fe77-464c-a04f-1bcecd25b6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:33.053131 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfaa98-fe77-464c-a04f-1bcecd25b6d4 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:33.053473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfaa98-fe77-464c-a04f-1bcecd25b6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:33.053815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfaa98-fe77-464c-a04f-1bcecd25b6d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:33.055093 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfaa98-fe77-464c-a04f-1bcecd25b6d4 None admin] RBAC: Adding query filter params (role.id=70ac549382b14b2cbb8b35e5dfc648d4, scope.project.id=9cd39464055141b8b43fde20d0885c99, user.id=6c9e458c2c5f4ed1b3eddd84c2ec1703) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:33.055506 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfaa98-fe77-464c-a04f-1bcecd25b6d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:33.057783 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfaa98-fe77-464c-a04f-1bcecd25b6d4 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:33.065582 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 654/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Mon May 9 09:37:32 2022] GET /identity/v3/role_assignments?user.id=6c9e458c2c5f4ed1b3eddd84c2ec1703&scope.project.id=9cd39464055141b8b43fde20d0885c99&role.id=70ac549382b14b2cbb8b35e5dfc648d4 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:35.265983 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb2e6f-a1c5-441f-af14-904095c727bd None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:35.266850 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb2e6f-a1c5-441f-af14-904095c727bd None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:35.266850 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb2e6f-a1c5-441f-af14-904095c727bd None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:35.269104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 634/1288] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:37:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:37:35.274471 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29ceaa-5dd8-4a98-9d56-dd3a3a82baf6 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:35.274898 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29ceaa-5dd8-4a98-9d56-dd3a3a82baf6 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:35.274989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29ceaa-5dd8-4a98-9d56-dd3a3a82baf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:35.292153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-3b29ceaa-5dd8-4a98-9d56-dd3a3a82baf6 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:37:35.293355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-3b29ceaa-5dd8-4a98-9d56-dd3a3a82baf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:37:35.358569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 655/1289] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:37:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:37:35.381042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb2e6f-a1c5-441f-af14-904095c727bd None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:35.493651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb2e6f-a1c5-441f-af14-904095c727bd None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:35.527578 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88524cb-3df8-419a-b979-157e82f642aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:35.528279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88524cb-3df8-419a-b979-157e82f642aa None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:35.528553 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88524cb-3df8-419a-b979-157e82f642aa None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:35.528851 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88524cb-3df8-419a-b979-157e82f642aa None admin] PATH_INFO: `/v3/services/placement` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:35.529497 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88524cb-3df8-419a-b979-157e82f642aa None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:35.530886 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88524cb-3df8-419a-b979-157e82f642aa None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:35.542848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-e88524cb-3df8-419a-b979-157e82f642aa None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 09:37:35.542848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:35.542848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:37:35.542848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:37:35.542848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:37:35.542848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:37:35.542848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:37:35.542848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:37:35.542848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 09:37:35.542848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:35.542848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:37:35.542848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:35.542848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:35.542848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:35.542848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:35.542848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:35.542848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:35.544061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:35.544061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:35.544061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:35.544061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:35.544061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:35.544061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:35.544061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:37:35.544061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:37:35.544061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:37:35.544061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:37:35.544061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:35.544061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:35.544061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:35.544061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:35.544061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:35.544061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:35.545635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:35.545635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:37:35.545635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:35.545635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:35.545635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:35.545635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:35.545635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:35.545635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:35.545635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:35.545635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:35.545635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:37:35.545635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:37:35.545635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 09:37:35.545635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:35.545635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 635/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Mon May 9 09:37:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:35.553519 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29ceaa-5dd8-4a98-9d56-dd3a3a82baf6 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:35.653891 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29ceaa-5dd8-4a98-9d56-dd3a3a82baf6 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:35.689267 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177c810-d0db-40d5-991f-b71aa8ce1ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:35.689923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177c810-d0db-40d5-991f-b71aa8ce1ec6 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:35.690195 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177c810-d0db-40d5-991f-b71aa8ce1ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:35.690463 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177c810-d0db-40d5-991f-b71aa8ce1ec6 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:35.691171 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177c810-d0db-40d5-991f-b71aa8ce1ec6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:35.691476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177c810-d0db-40d5-991f-b71aa8ce1ec6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:35.692873 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177c810-d0db-40d5-991f-b71aa8ce1ec6 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:35.698363 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 656/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Mon May 9 09:37:35 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:35.704886 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88524cb-3df8-419a-b979-157e82f642aa None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:35.823887 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88524cb-3df8-419a-b979-157e82f642aa None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:35.856385 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3207a4-061e-4d73-af9d-4e29dfb7efd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:35.857064 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3207a4-061e-4d73-af9d-4e29dfb7efd8 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:35.857338 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3207a4-061e-4d73-af9d-4e29dfb7efd8 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:35.857609 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3207a4-061e-4d73-af9d-4e29dfb7efd8 None admin] PATH_INFO: `/v3/services` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:35.858311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3207a4-061e-4d73-af9d-4e29dfb7efd8 None admin] RBAC: Adding query filter params (type=placement) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:35.858612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3207a4-061e-4d73-af9d-4e29dfb7efd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:35.860017 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3207a4-061e-4d73-af9d-4e29dfb7efd8 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:35.864485 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 636/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Mon May 9 09:37:35 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:38.047803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6da44-80f0-41a0-87f3-d3df8ac9e98d None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:38.048643 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6da44-80f0-41a0-87f3-d3df8ac9e98d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:38.048643 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6da44-80f0-41a0-87f3-d3df8ac9e98d None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:38.050937 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 657/1293] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:37:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:37:38.056311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a83d9-2840-4b7c-b46f-7ad78bc1d28e None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:38.056754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a83d9-2840-4b7c-b46f-7ad78bc1d28e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:38.056862 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a83d9-2840-4b7c-b46f-7ad78bc1d28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:38.075921 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-b73a83d9-2840-4b7c-b46f-7ad78bc1d28e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:37:38.077134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-b73a83d9-2840-4b7c-b46f-7ad78bc1d28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:37:38.150807 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 637/1294] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:37:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:37:38.178644 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce6da44-80f0-41a0-87f3-d3df8ac9e98d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:38.304930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce6da44-80f0-41a0-87f3-d3df8ac9e98d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:38.338665 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39edc7b3-650f-4a8b-9442-c1a8ce93b606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:38.339305 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edc7b3-650f-4a8b-9442-c1a8ce93b606 None admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:38.339577 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edc7b3-650f-4a8b-9442-c1a8ce93b606 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:38.339845 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edc7b3-650f-4a8b-9442-c1a8ce93b606 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:38.340554 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39edc7b3-650f-4a8b-9442-c1a8ce93b606 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:38.341942 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39edc7b3-650f-4a8b-9442-c1a8ce93b606 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:38.362802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 658/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Mon May 9 09:37:38 2022] POST /identity/v3/services => generated 246 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:37:40.591001 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8556a09-0e3d-487a-bae8-10571864139c None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:40.591510 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8556a09-0e3d-487a-bae8-10571864139c None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:40.591510 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8556a09-0e3d-487a-bae8-10571864139c None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:40.592969 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 638/1296] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:37:40.599361 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c141d-e766-44b2-8b72-a81aa16227a6 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:40.600064 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c141d-e766-44b2-8b72-a81aa16227a6 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:40.600203 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c141d-e766-44b2-8b72-a81aa16227a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:40.626775 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-628c141d-e766-44b2-8b72-a81aa16227a6 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:37:40.627971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-628c141d-e766-44b2-8b72-a81aa16227a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:37:40.701288 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 659/1297] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:37:40 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:37:40.724215 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8556a09-0e3d-487a-bae8-10571864139c None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:40.834237 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8556a09-0e3d-487a-bae8-10571864139c None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:40.868427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6715c-8aea-4d28-8b85-a27d94b2b87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:40.869241 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6715c-8aea-4d28-8b85-a27d94b2b87d None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:40.869511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6715c-8aea-4d28-8b85-a27d94b2b87d None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:40.869777 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6715c-8aea-4d28-8b85-a27d94b2b87d None admin] PATH_INFO: `/v3/services/placement` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:40.870420 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc6715c-8aea-4d28-8b85-a27d94b2b87d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:40.871795 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc6715c-8aea-4d28-8b85-a27d94b2b87d None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:40.883657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-9cc6715c-8aea-4d28-8b85-a27d94b2b87d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 09:37:40.883657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:40.883657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:37:40.883657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:37:40.883657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:37:40.883657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:37:40.883657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:37:40.883657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:37:40.883657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 09:37:40.883657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:40.883657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:37:40.883657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:40.883657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:40.883657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:40.883657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:40.883657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:40.883657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:40.884831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:40.884831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:40.884831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:40.884831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:40.884831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:40.884831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:40.884831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:37:40.884831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:37:40.884831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:37:40.884831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:37:40.884831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:40.884831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:40.884831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:40.884831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:40.884831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:40.884831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:40.885812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:40.885812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:37:40.885812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:40.885812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:40.885812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:40.885812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:40.885812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:40.885812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:40.885812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:40.885812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:40.885812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:37:40.885812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:37:40.885812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 09:37:40.885812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:40.885812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 639/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Mon May 9 09:37:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:40.891349 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c141d-e766-44b2-8b72-a81aa16227a6 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:40.990812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c141d-e766-44b2-8b72-a81aa16227a6 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:41.024485 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff704ed-e4f6-4aad-be76-07b01408e168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:41.025174 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff704ed-e4f6-4aad-be76-07b01408e168 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:41.025458 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff704ed-e4f6-4aad-be76-07b01408e168 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:41.025727 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff704ed-e4f6-4aad-be76-07b01408e168 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:41.026397 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff704ed-e4f6-4aad-be76-07b01408e168 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:41.026729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff704ed-e4f6-4aad-be76-07b01408e168 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:41.028098 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff704ed-e4f6-4aad-be76-07b01408e168 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:41.033136 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 660/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Mon May 9 09:37:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:41.039187 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6715c-8aea-4d28-8b85-a27d94b2b87d None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:41.142030 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6715c-8aea-4d28-8b85-a27d94b2b87d None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:41.175433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155beac4-36ff-458d-b6e7-e33618abb0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:41.176061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155beac4-36ff-458d-b6e7-e33618abb0e5 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:41.176337 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155beac4-36ff-458d-b6e7-e33618abb0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:41.176604 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155beac4-36ff-458d-b6e7-e33618abb0e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:41.177417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155beac4-36ff-458d-b6e7-e33618abb0e5 None admin] RBAC: Adding query filter params (interface=public, service_id=5e3fcf51372844b6914d8cade870a715, region_id=RegionOne) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:41.177728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155beac4-36ff-458d-b6e7-e33618abb0e5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:41.179065 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155beac4-36ff-458d-b6e7-e33618abb0e5 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:41.184160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 640/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Mon May 9 09:37:41 2022] GET /identity/v3/endpoints?service_id=5e3fcf51372844b6914d8cade870a715&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:41.190035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff704ed-e4f6-4aad-be76-07b01408e168 None admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:41.308606 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff704ed-e4f6-4aad-be76-07b01408e168 None admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:41.341916 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe2247-5931-446c-8c8c-060796ac20ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:41.342560 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe2247-5931-446c-8c8c-060796ac20ab None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:41.342836 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe2247-5931-446c-8c8c-060796ac20ab None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:41.343153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe2247-5931-446c-8c8c-060796ac20ab None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:41.343819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe2247-5931-446c-8c8c-060796ac20ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:41.345215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe2247-5931-446c-8c8c-060796ac20ab None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:41.351892 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 661/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Mon May 9 09:37:41 2022] GET /identity/v3/services => generated 2191 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:37:43.529665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad8bde-1c2f-4f26-aaf2-889fc0501ace None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:43.530605 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad8bde-1c2f-4f26-aaf2-889fc0501ace None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:43.530605 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad8bde-1c2f-4f26-aaf2-889fc0501ace None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:43.532008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 641/1302] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:37:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:37:43.537492 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee11ad-f20d-4e73-98aa-2c4d9ce72f94 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:43.537921 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee11ad-f20d-4e73-98aa-2c4d9ce72f94 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:43.538009 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee11ad-f20d-4e73-98aa-2c4d9ce72f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:43.561159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-e9ee11ad-f20d-4e73-98aa-2c4d9ce72f94 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:37:43.562691 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e9ee11ad-f20d-4e73-98aa-2c4d9ce72f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:37:43.628944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 662/1303] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:37:43 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 09:37:43.652190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad8bde-1c2f-4f26-aaf2-889fc0501ace None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:43.757072 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad8bde-1c2f-4f26-aaf2-889fc0501ace None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:43.790718 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d3a36-1dd7-4b15-8f94-ce5acb2e14c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:43.791505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d3a36-1dd7-4b15-8f94-ce5acb2e14c1 None admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:43.791779 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d3a36-1dd7-4b15-8f94-ce5acb2e14c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:43.792042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d3a36-1dd7-4b15-8f94-ce5acb2e14c1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:43.792716 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d3a36-1dd7-4b15-8f94-ce5acb2e14c1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:43.794080 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d3a36-1dd7-4b15-8f94-ce5acb2e14c1 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:43.806031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application [None req-bc1d3a36-1dd7-4b15-8f94-ce5acb2e14c1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 09:37:43.806031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:43.806031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 09:37:43.806031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 09:37:43.806031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 09:37:43.806031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 09:37:43.806031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 09:37:43.806031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:37:43.806031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 09:37:43.806031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:43.806031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 09:37:43.806031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:43.806031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 09:37:43.806031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 09:37:43.806031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 09:37:43.806031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 09:37:43.806031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 09:37:43.807251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 09:37:43.807251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 09:37:43.807251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 09:37:43.807251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 09:37:43.807251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 09:37:43.807251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 09:37:43.807251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 09:37:43.807251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 09:37:43.807251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 09:37:43.807251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 09:37:43.807251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 09:37:43.807251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 09:37:43.807251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 09:37:43.807251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 09:37:43.807251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 09:37:43.807251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.get_or_create( May 09 09:37:43.808506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 09:37:43.808506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application with Lock( May 09 09:37:43.808506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 09:37:43.808506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self._enter() May 09 09:37:43.808506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 09:37:43.808506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 09:37:43.808506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 09:37:43.808506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application return self.creator() May 09 09:37:43.808506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 09:37:43.808506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application created_value = creator( May 09 09:37:43.808506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 09:37:43.808506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 09:37:43.808506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 09:37:43.808506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: ERROR keystone.server.flask.application May 09 09:37:43.808506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 642/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Mon May 9 09:37:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 09:37:43.814317 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee11ad-f20d-4e73-98aa-2c4d9ce72f94 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:43.925093 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee11ad-f20d-4e73-98aa-2c4d9ce72f94 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:43.958944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc93f74-ee9a-49f8-b1b5-ca651fbd94e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:43.959588 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc93f74-ee9a-49f8-b1b5-ca651fbd94e0 None admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:43.959863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc93f74-ee9a-49f8-b1b5-ca651fbd94e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:43.960135 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc93f74-ee9a-49f8-b1b5-ca651fbd94e0 None admin] PATH_INFO: `/v3/services` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:43.960871 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc93f74-ee9a-49f8-b1b5-ca651fbd94e0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:37:43.961183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc93f74-ee9a-49f8-b1b5-ca651fbd94e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:43.962553 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc93f74-ee9a-49f8-b1b5-ca651fbd94e0 None admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:43.967699 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 663/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Mon May 9 09:37:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:37:43.973680 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d3a36-1dd7-4b15-8f94-ce5acb2e14c1 None admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:37:44.076232 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d3a36-1dd7-4b15-8f94-ce5acb2e14c1 None admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:37:44.109663 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3604ac3-aac2-4075-91b4-28e6b8038d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:37:44.110298 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3604ac3-aac2-4075-91b4-28e6b8038d95 None admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:37:44.110559 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3604ac3-aac2-4075-91b4-28e6b8038d95 None admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:37:44.110833 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3604ac3-aac2-4075-91b4-28e6b8038d95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:37:44.111572 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3604ac3-aac2-4075-91b4-28e6b8038d95 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:37:44.112972 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3604ac3-aac2-4075-91b4-28e6b8038d95 None admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:37:44.134773 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 643/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Mon May 9 09:37:43 2022] POST /identity/v3/endpoints => generated 331 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:38:47.405678 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c79cb0-b9c3-4e91-b502-e902a597cc3b None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:38:47.406708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c79cb0-b9c3-4e91-b502-e902a597cc3b None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:38:47.407195 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c79cb0-b9c3-4e91-b502-e902a597cc3b None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:38:47.413036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 664/1307] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:38:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:38:47.416634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b29aba-2c57-42d0-83c8-27ea53929f69 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:38:47.417631 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b29aba-2c57-42d0-83c8-27ea53929f69 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:38:47.417631 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b29aba-2c57-42d0-83c8-27ea53929f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:38:47.566047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-f1b29aba-2c57-42d0-83c8-27ea53929f69 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:38:47.567295 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f1b29aba-2c57-42d0-83c8-27ea53929f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:38:47.748340 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 644/1308] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:38:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:38:47.816191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784cfd6-9e5e-4ccb-93ff-156414a32bc4 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:38:47.816451 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784cfd6-9e5e-4ccb-93ff-156414a32bc4 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:38:47.816753 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784cfd6-9e5e-4ccb-93ff-156414a32bc4 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:38:47.818149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 665/1309] 10.222.0.26 () {54 vars in 964 bytes} [Mon May 9 09:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:38:47.827384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96df7b-42bb-4a4a-a16e-1ec775ed3bdf None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:38:47.827748 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96df7b-42bb-4a4a-a16e-1ec775ed3bdf None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:38:47.828100 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96df7b-42bb-4a4a-a16e-1ec775ed3bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:38:47.914288 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-9d96df7b-42bb-4a4a-a16e-1ec775ed3bdf None None] MFA Rules not processed for user `0f0d3f4264a04093894d6bc5c5c45c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:38:47.915507 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-9d96df7b-42bb-4a4a-a16e-1ec775ed3bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:38:48.017632 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 645/1310] 10.222.0.26 () {58 vars in 1061 bytes} [Mon May 9 09:38:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:38:48.027584 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784cfd6-9e5e-4ccb-93ff-156414a32bc4 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:38:48.029494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-9784cfd6-9e5e-4ccb-93ff-156414a32bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:38:48.158481 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784cfd6-9e5e-4ccb-93ff-156414a32bc4 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:38:48.235363 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7e58d-0d6c-4212-a2a6-e2496b056995 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:38:48.236079 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7e58d-0d6c-4212-a2a6-e2496b056995 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:38:48.236366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7e58d-0d6c-4212-a2a6-e2496b056995 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:38:48.236639 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7e58d-0d6c-4212-a2a6-e2496b056995 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:38:48.238812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-2fe7e58d-0d6c-4212-a2a6-e2496b056995 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:38:48.354879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7e58d-0d6c-4212-a2a6-e2496b056995 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:38:48.356333 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7e58d-0d6c-4212-a2a6-e2496b056995 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:38:48.376410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 666/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:38:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:38:50.944101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3493944-89dc-4846-8b9e-d878bfa6db1e None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:38:50.944641 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3493944-89dc-4846-8b9e-d878bfa6db1e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:38:50.944641 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3493944-89dc-4846-8b9e-d878bfa6db1e None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:38:50.946059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 646/1312] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:38:50.951568 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470bd8d-4aad-495e-a419-a522763010fe None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:38:50.952109 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470bd8d-4aad-495e-a419-a522763010fe None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:38:50.952109 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470bd8d-4aad-495e-a419-a522763010fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:38:51.032487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-d470bd8d-4aad-495e-a419-a522763010fe None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:38:51.033732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-d470bd8d-4aad-495e-a419-a522763010fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:38:51.127550 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 667/1313] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:38:51.180603 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b9d63-f9f4-4d1c-bb29-8c6f2792d548 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:38:51.181008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b9d63-f9f4-4d1c-bb29-8c6f2792d548 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:38:51.181340 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b9d63-f9f4-4d1c-bb29-8c6f2792d548 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:38:51.182929 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 647/1314] 10.222.0.26 () {54 vars in 964 bytes} [Mon May 9 09:38:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:38:51.189390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906f541-efd3-4a1d-a6e5-b2afecb4b028 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:38:51.189674 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906f541-efd3-4a1d-a6e5-b2afecb4b028 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:38:51.189938 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906f541-efd3-4a1d-a6e5-b2afecb4b028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:38:51.267012 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-f906f541-efd3-4a1d-a6e5-b2afecb4b028 None None] MFA Rules not processed for user `0f0d3f4264a04093894d6bc5c5c45c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:38:51.268120 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-f906f541-efd3-4a1d-a6e5-b2afecb4b028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:38:51.341904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 668/1315] 10.222.0.26 () {58 vars in 1061 bytes} [Mon May 9 09:38:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:38:51.350934 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b9d63-f9f4-4d1c-bb29-8c6f2792d548 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:38:51.352438 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-752b9d63-f9f4-4d1c-bb29-8c6f2792d548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:38:51.453521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b9d63-f9f4-4d1c-bb29-8c6f2792d548 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:38:51.528383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa25ca5-19d3-42df-a22e-8eadf8784e45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:38:51.529113 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa25ca5-19d3-42df-a22e-8eadf8784e45 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:38:51.529366 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa25ca5-19d3-42df-a22e-8eadf8784e45 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:38:51.529648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa25ca5-19d3-42df-a22e-8eadf8784e45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:38:51.531305 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-9aa25ca5-19d3-42df-a22e-8eadf8784e45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:38:51.638869 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa25ca5-19d3-42df-a22e-8eadf8784e45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:38:51.640221 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa25ca5-19d3-42df-a22e-8eadf8784e45 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:38:51.658781 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 648/1316] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:38:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:38:54.385659 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c25732-8422-4eb3-8025-f8c72e3ce9c0 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:38:54.386365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c25732-8422-4eb3-8025-f8c72e3ce9c0 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:38:54.386365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c25732-8422-4eb3-8025-f8c72e3ce9c0 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:38:54.387590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 669/1317] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:38:54.394097 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45416df8-600d-4a5c-99a0-c94e60762d8a None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:38:54.394573 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45416df8-600d-4a5c-99a0-c94e60762d8a None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:38:54.394679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45416df8-600d-4a5c-99a0-c94e60762d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:38:54.470908 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-45416df8-600d-4a5c-99a0-c94e60762d8a None None] MFA Rules not processed for user `7683dcda9e5649fc8b526a0067af2d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:38:54.472112 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-45416df8-600d-4a5c-99a0-c94e60762d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:38:54.566515 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 649/1318] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:38:54 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:38:54.605804 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c25732-8422-4eb3-8025-f8c72e3ce9c0 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:38:54.607355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-97c25732-8422-4eb3-8025-f8c72e3ce9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:38:54.724840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c25732-8422-4eb3-8025-f8c72e3ce9c0 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:38:54.798898 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673df24-765d-4ae4-a878-a73efa2cf26d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:38:54.799537 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673df24-765d-4ae4-a878-a73efa2cf26d service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:38:54.799815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673df24-765d-4ae4-a878-a73efa2cf26d service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:38:54.800084 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673df24-765d-4ae4-a878-a73efa2cf26d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:38:54.801776 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-d673df24-765d-4ae4-a878-a73efa2cf26d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:38:54.922307 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d673df24-765d-4ae4-a878-a73efa2cf26d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:38:54.923753 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d673df24-765d-4ae4-a878-a73efa2cf26d service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:38:54.944797 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 670/1319] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:38:54 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:38:57.710198 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c92b6a-4390-4a0b-9771-d5d26fd70a18 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:38:57.710735 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c92b6a-4390-4a0b-9771-d5d26fd70a18 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:38:57.710735 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c92b6a-4390-4a0b-9771-d5d26fd70a18 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:38:57.712299 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 650/1320] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:38:57.721706 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916b037-8822-4a24-a50b-52e78c574508 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:38:57.722833 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916b037-8822-4a24-a50b-52e78c574508 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:38:57.722833 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916b037-8822-4a24-a50b-52e78c574508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:38:57.788557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-9916b037-8822-4a24-a50b-52e78c574508 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:38:57.789793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-9916b037-8822-4a24-a50b-52e78c574508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:38:57.883611 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 671/1321] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:38:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:38:57.927722 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c92b6a-4390-4a0b-9771-d5d26fd70a18 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:38:57.929267 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-56c92b6a-4390-4a0b-9771-d5d26fd70a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:38:58.040442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c92b6a-4390-4a0b-9771-d5d26fd70a18 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:38:58.123985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24d4d5-3a51-4a3e-b017-88e672bc630a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:38:58.124625 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24d4d5-3a51-4a3e-b017-88e672bc630a service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:38:58.124937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24d4d5-3a51-4a3e-b017-88e672bc630a service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:38:58.125203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24d4d5-3a51-4a3e-b017-88e672bc630a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:38:58.126837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-ad24d4d5-3a51-4a3e-b017-88e672bc630a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:38:58.236116 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24d4d5-3a51-4a3e-b017-88e672bc630a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:38:58.237879 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24d4d5-3a51-4a3e-b017-88e672bc630a service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:38:58.261581 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 651/1322] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:38:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:39:01.121158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b888b9e-c982-4776-89b5-b2d198b652fc None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:01.121751 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b888b9e-c982-4776-89b5-b2d198b652fc None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:01.121751 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b888b9e-c982-4776-89b5-b2d198b652fc None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:01.123104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 672/1323] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:39:01.128452 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274e2d0-621e-4b7d-ad57-4b28b9e9a8d8 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:01.128958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274e2d0-621e-4b7d-ad57-4b28b9e9a8d8 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:01.129068 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274e2d0-621e-4b7d-ad57-4b28b9e9a8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:01.193153 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-2274e2d0-621e-4b7d-ad57-4b28b9e9a8d8 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:01.194366 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-2274e2d0-621e-4b7d-ad57-4b28b9e9a8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:01.290487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 652/1324] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:39:01.327237 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b888b9e-c982-4776-89b5-b2d198b652fc None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:39:01.328870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-2b888b9e-c982-4776-89b5-b2d198b652fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:01.432580 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b888b9e-c982-4776-89b5-b2d198b652fc None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:39:01.506398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a18ad0-a791-49ae-9dea-6b0306c75e61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:39:01.507041 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a18ad0-a791-49ae-9dea-6b0306c75e61 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:01.507317 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a18ad0-a791-49ae-9dea-6b0306c75e61 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:01.507585 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a18ad0-a791-49ae-9dea-6b0306c75e61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:01.509256 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-10a18ad0-a791-49ae-9dea-6b0306c75e61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:01.622239 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a18ad0-a791-49ae-9dea-6b0306c75e61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:39:01.623709 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a18ad0-a791-49ae-9dea-6b0306c75e61 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:39:01.643116 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 673/1325] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:39:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:39:03.888067 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8872ec5-5fcf-47c8-86a7-f793efc980a7 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:03.888638 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8872ec5-5fcf-47c8-86a7-f793efc980a7 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:03.888638 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8872ec5-5fcf-47c8-86a7-f793efc980a7 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:03.890004 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 653/1326] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:39:03.895340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0b14d-d309-4a3e-9974-0bbfa1ac678d None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:03.895810 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0b14d-d309-4a3e-9974-0bbfa1ac678d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:03.895892 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0b14d-d309-4a3e-9974-0bbfa1ac678d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:03.957343 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-00e0b14d-d309-4a3e-9974-0bbfa1ac678d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:03.958540 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-00e0b14d-d309-4a3e-9974-0bbfa1ac678d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:04.054945 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 674/1327] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:39:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:39:04.089357 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8872ec5-5fcf-47c8-86a7-f793efc980a7 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:39:04.090852 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-c8872ec5-5fcf-47c8-86a7-f793efc980a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:04.190719 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8872ec5-5fcf-47c8-86a7-f793efc980a7 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:39:04.262890 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42373c8c-d4ee-49d3-a5f2-1d975295bd5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:39:04.263530 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42373c8c-d4ee-49d3-a5f2-1d975295bd5c service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:04.263804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42373c8c-d4ee-49d3-a5f2-1d975295bd5c service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:04.264071 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42373c8c-d4ee-49d3-a5f2-1d975295bd5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:04.265725 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-42373c8c-d4ee-49d3-a5f2-1d975295bd5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:04.372678 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42373c8c-d4ee-49d3-a5f2-1d975295bd5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:39:04.374137 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42373c8c-d4ee-49d3-a5f2-1d975295bd5c service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:39:04.392467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 654/1328] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:39:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:39:06.629112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa9670-112d-412f-9e3b-4dad46fa00a6 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:06.629732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa9670-112d-412f-9e3b-4dad46fa00a6 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:06.629732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa9670-112d-412f-9e3b-4dad46fa00a6 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:06.631078 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 675/1329] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:39:06.636645 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2a30b-af78-48f6-92ff-8b6f41157a72 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:06.637140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2a30b-af78-48f6-92ff-8b6f41157a72 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:06.637245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2a30b-af78-48f6-92ff-8b6f41157a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:06.708120 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-dac2a30b-af78-48f6-92ff-8b6f41157a72 None None] MFA Rules not processed for user `7683dcda9e5649fc8b526a0067af2d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:06.709451 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-dac2a30b-af78-48f6-92ff-8b6f41157a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:06.806845 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 655/1330] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:39:06 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:39:06.844306 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa9670-112d-412f-9e3b-4dad46fa00a6 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:39:06.845928 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c2aa9670-112d-412f-9e3b-4dad46fa00a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:06.946162 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa9670-112d-412f-9e3b-4dad46fa00a6 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:39:07.019429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09127e3-1445-4703-805e-dba209a73092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:39:07.020082 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09127e3-1445-4703-805e-dba209a73092 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:07.020365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09127e3-1445-4703-805e-dba209a73092 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:07.020642 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09127e3-1445-4703-805e-dba209a73092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:07.022336 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c09127e3-1445-4703-805e-dba209a73092 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:07.129789 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09127e3-1445-4703-805e-dba209a73092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:39:07.131146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09127e3-1445-4703-805e-dba209a73092 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:39:07.150451 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 676/1331] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:39:06 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:39:09.728525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0cf68-ea8e-465a-b648-0cb2b0ea7f7c None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:09.729159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0cf68-ea8e-465a-b648-0cb2b0ea7f7c None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:09.729159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0cf68-ea8e-465a-b648-0cb2b0ea7f7c None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:09.730427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 656/1332] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:39:09.737805 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9247516-72fc-47c2-b6f3-46c059061e7d None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:09.738305 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9247516-72fc-47c2-b6f3-46c059061e7d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:09.738660 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9247516-72fc-47c2-b6f3-46c059061e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:09.815338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-c9247516-72fc-47c2-b6f3-46c059061e7d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:09.816487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c9247516-72fc-47c2-b6f3-46c059061e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:09.909962 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 677/1333] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:39:09.945186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0cf68-ea8e-465a-b648-0cb2b0ea7f7c None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:39:09.946696 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-cca0cf68-ea8e-465a-b648-0cb2b0ea7f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:10.055098 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0cf68-ea8e-465a-b648-0cb2b0ea7f7c None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:39:10.131743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d14cd7-eb66-4b46-9844-577e23a24f57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:39:10.132392 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d14cd7-eb66-4b46-9844-577e23a24f57 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:10.132694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d14cd7-eb66-4b46-9844-577e23a24f57 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:10.132957 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d14cd7-eb66-4b46-9844-577e23a24f57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:10.134649 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e6d14cd7-eb66-4b46-9844-577e23a24f57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:10.246029 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d14cd7-eb66-4b46-9844-577e23a24f57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:39:10.247519 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d14cd7-eb66-4b46-9844-577e23a24f57 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:39:10.265839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 657/1334] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:39:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 09:39:12.491084 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219482b7-0187-4dab-8f67-b7e9f611013e None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:12.492009 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219482b7-0187-4dab-8f67-b7e9f611013e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:12.492009 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219482b7-0187-4dab-8f67-b7e9f611013e None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:12.494200 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 678/1335] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:39:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:39:12.502300 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62688f-e995-4b56-a894-39180a2526a1 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:12.502766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62688f-e995-4b56-a894-39180a2526a1 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:12.502861 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62688f-e995-4b56-a894-39180a2526a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:12.567931 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-0d62688f-e995-4b56-a894-39180a2526a1 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:12.569146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-0d62688f-e995-4b56-a894-39180a2526a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:12.664131 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 658/1336] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:39:12.717705 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219482b7-0187-4dab-8f67-b7e9f611013e None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:39:12.719836 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-219482b7-0187-4dab-8f67-b7e9f611013e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:12.832516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219482b7-0187-4dab-8f67-b7e9f611013e None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:39:12.921849 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5788e-395f-4a7c-b8c3-8d63e5846bff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:39:12.922482 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5788e-395f-4a7c-b8c3-8d63e5846bff service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:12.922755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5788e-395f-4a7c-b8c3-8d63e5846bff service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:12.923021 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5788e-395f-4a7c-b8c3-8d63e5846bff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:12.924637 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c6c5788e-395f-4a7c-b8c3-8d63e5846bff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:13.045059 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5788e-395f-4a7c-b8c3-8d63e5846bff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:39:13.046494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5788e-395f-4a7c-b8c3-8d63e5846bff service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:39:13.065015 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 679/1337] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:39:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:39:15.718983 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cbebd-9326-4bea-aa1c-37a6ed93fee2 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:15.719517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cbebd-9326-4bea-aa1c-37a6ed93fee2 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:15.719517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cbebd-9326-4bea-aa1c-37a6ed93fee2 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:15.720867 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 659/1338] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:39:15.726950 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ff060-4b07-4fac-9732-5ff63d03d11d None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:15.727631 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ff060-4b07-4fac-9732-5ff63d03d11d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:15.727871 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ff060-4b07-4fac-9732-5ff63d03d11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:15.794568 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-990ff060-4b07-4fac-9732-5ff63d03d11d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:15.795817 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-990ff060-4b07-4fac-9732-5ff63d03d11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:15.908737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 680/1339] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:39:15 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:39:15.947514 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cbebd-9326-4bea-aa1c-37a6ed93fee2 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:39:15.950134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-723cbebd-9326-4bea-aa1c-37a6ed93fee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:16.064391 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cbebd-9326-4bea-aa1c-37a6ed93fee2 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:39:16.139463 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efca12a-e6d9-431b-b591-1e659a9204d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:39:16.140110 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efca12a-e6d9-431b-b591-1e659a9204d4 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:16.140385 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efca12a-e6d9-431b-b591-1e659a9204d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:16.140652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efca12a-e6d9-431b-b591-1e659a9204d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:16.142308 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-1efca12a-e6d9-431b-b591-1e659a9204d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:16.253364 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efca12a-e6d9-431b-b591-1e659a9204d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:39:16.254809 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efca12a-e6d9-431b-b591-1e659a9204d4 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:39:16.272696 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 660/1340] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:39:15 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:39:20.561445 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07612243-9339-4e1d-90a8-d76f78fff641 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:20.561886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07612243-9339-4e1d-90a8-d76f78fff641 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:20.562011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07612243-9339-4e1d-90a8-d76f78fff641 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:20.563434 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 681/1341] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:39:20.568853 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a02834-6a44-4dac-9311-766b1c1183fa None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:20.569321 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a02834-6a44-4dac-9311-766b1c1183fa None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:20.569417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a02834-6a44-4dac-9311-766b1c1183fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:20.643395 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-e9a02834-6a44-4dac-9311-766b1c1183fa None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:20.644605 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e9a02834-6a44-4dac-9311-766b1c1183fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:20.751340 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 661/1342] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:39:20.789479 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07612243-9339-4e1d-90a8-d76f78fff641 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:39:20.791962 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-07612243-9339-4e1d-90a8-d76f78fff641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:20.889310 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07612243-9339-4e1d-90a8-d76f78fff641 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:39:20.892972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95215ae5-34e8-43d6-b351-af3fd62cde69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:39:20.893620 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95215ae5-34e8-43d6-b351-af3fd62cde69 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:20.893900 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95215ae5-34e8-43d6-b351-af3fd62cde69 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:20.894165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95215ae5-34e8-43d6-b351-af3fd62cde69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:20.903439 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95215ae5-34e8-43d6-b351-af3fd62cde69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:39:20.904917 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95215ae5-34e8-43d6-b351-af3fd62cde69 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:39:20.914321 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 682/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:39:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:39:23.691077 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fd124-d4db-4bb2-992c-795ca6cee9a0 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:23.691602 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fd124-d4db-4bb2-992c-795ca6cee9a0 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:23.691602 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fd124-d4db-4bb2-992c-795ca6cee9a0 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:23.693134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 662/1344] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:39:23.698625 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecdb48-feca-4962-9678-1f56b955e915 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:23.699072 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecdb48-feca-4962-9678-1f56b955e915 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:23.699209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecdb48-feca-4962-9678-1f56b955e915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:23.729882 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-cdecdb48-feca-4962-9678-1f56b955e915 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:23.731247 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-cdecdb48-feca-4962-9678-1f56b955e915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:23.813200 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 683/1345] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:39:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:39:23.849759 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fd124-d4db-4bb2-992c-795ca6cee9a0 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:39:23.861539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fd124-d4db-4bb2-992c-795ca6cee9a0 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:39:23.865124 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e042f3-2bc2-44e8-a780-ba5726f36c22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:39:23.865759 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e042f3-2bc2-44e8-a780-ba5726f36c22 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:23.866034 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e042f3-2bc2-44e8-a780-ba5726f36c22 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:23.866304 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e042f3-2bc2-44e8-a780-ba5726f36c22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:23.875341 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e042f3-2bc2-44e8-a780-ba5726f36c22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:39:23.876807 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e042f3-2bc2-44e8-a780-ba5726f36c22 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:39:23.886070 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 663/1346] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:39:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:39:28.155607 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f1583-96e2-41cb-8404-b68907cefbd0 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:28.156181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f1583-96e2-41cb-8404-b68907cefbd0 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:28.156181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f1583-96e2-41cb-8404-b68907cefbd0 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:28.157458 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 684/1347] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:39:28.163100 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da033e8-c31d-4bf5-92e1-7a5d6628c6ed None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:28.163593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da033e8-c31d-4bf5-92e1-7a5d6628c6ed None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:28.163696 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da033e8-c31d-4bf5-92e1-7a5d6628c6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:28.183175 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-3da033e8-c31d-4bf5-92e1-7a5d6628c6ed None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:28.184366 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-3da033e8-c31d-4bf5-92e1-7a5d6628c6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:28.192786 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 664/1348] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:39:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:39:28.230786 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f1583-96e2-41cb-8404-b68907cefbd0 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:39:28.232879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-f13f1583-96e2-41cb-8404-b68907cefbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:28.247791 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f1583-96e2-41cb-8404-b68907cefbd0 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:39:28.251652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0dee8-136a-44bd-b969-baa3776b6da6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:39:28.252282 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0dee8-136a-44bd-b969-baa3776b6da6 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:28.252556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0dee8-136a-44bd-b969-baa3776b6da6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:28.252856 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0dee8-136a-44bd-b969-baa3776b6da6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:28.262270 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c0dee8-136a-44bd-b969-baa3776b6da6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:39:28.263762 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c0dee8-136a-44bd-b969-baa3776b6da6 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:39:28.273020 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 685/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:39:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:39:33.544736 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c929c-ed17-453c-8a7f-7c5b0fa5e31a None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:33.545435 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c929c-ed17-453c-8a7f-7c5b0fa5e31a None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:33.545851 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c929c-ed17-453c-8a7f-7c5b0fa5e31a None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:33.547372 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 665/1350] 10.222.0.26 () {54 vars in 921 bytes} [Mon May 9 09:39:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:39:33.563616 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bec2dd-00e4-42e0-8e79-51ff1d850acc None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:33.564557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bec2dd-00e4-42e0-8e79-51ff1d850acc None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:33.565041 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bec2dd-00e4-42e0-8e79-51ff1d850acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:33.636349 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-21bec2dd-00e4-42e0-8e79-51ff1d850acc None None] MFA Rules not processed for user `6c9e458c2c5f4ed1b3eddd84c2ec1703`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:33.637562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-21bec2dd-00e4-42e0-8e79-51ff1d850acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:33.724129 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 686/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Mon May 9 09:39:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:39:37.822432 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef1003-5126-46cc-bcdf-0970e028e9c1 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:37.823395 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef1003-5126-46cc-bcdf-0970e028e9c1 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:37.823599 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef1003-5126-46cc-bcdf-0970e028e9c1 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:37.825369 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 666/1352] 10.222.0.26 () {54 vars in 921 bytes} [Mon May 9 09:39:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:39:37.832236 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99f78c-cc36-4f07-b2e3-d5dd2e91852e None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:37.832801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99f78c-cc36-4f07-b2e3-d5dd2e91852e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:37.832892 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99f78c-cc36-4f07-b2e3-d5dd2e91852e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:37.852232 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-9b99f78c-cc36-4f07-b2e3-d5dd2e91852e None None] MFA Rules not processed for user `6c9e458c2c5f4ed1b3eddd84c2ec1703`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:37.853628 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-9b99f78c-cc36-4f07-b2e3-d5dd2e91852e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:37.863044 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 687/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Mon May 9 09:39:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:39:39.550755 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb68198-81e4-470f-81d7-fc03d4cecd6a None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:39.551297 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb68198-81e4-470f-81d7-fc03d4cecd6a None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:39.551297 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb68198-81e4-470f-81d7-fc03d4cecd6a None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:39.552642 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 667/1354] 10.222.0.26 () {54 vars in 921 bytes} [Mon May 9 09:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:39:39.557714 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8936cf-0fed-4a12-afa3-6a7908d7cd54 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:39.558171 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8936cf-0fed-4a12-afa3-6a7908d7cd54 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:39.558395 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8936cf-0fed-4a12-afa3-6a7908d7cd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:39.577483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-1d8936cf-0fed-4a12-afa3-6a7908d7cd54 None None] MFA Rules not processed for user `6c9e458c2c5f4ed1b3eddd84c2ec1703`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:39.578657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-1d8936cf-0fed-4a12-afa3-6a7908d7cd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:39.587990 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 688/1355] 10.222.0.26 () {58 vars in 1018 bytes} [Mon May 9 09:39:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:39:43.348280 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac59ddd-4dcc-4a7f-9368-3b939c31ab31 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:43.348876 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac59ddd-4dcc-4a7f-9368-3b939c31ab31 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:43.348876 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac59ddd-4dcc-4a7f-9368-3b939c31ab31 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:43.350258 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 668/1356] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:39:43.355597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8568a-52f2-4323-babb-e67aeffd7811 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:43.356028 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8568a-52f2-4323-babb-e67aeffd7811 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:43.356134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8568a-52f2-4323-babb-e67aeffd7811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:43.374117 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-99a8568a-52f2-4323-babb-e67aeffd7811 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:43.375279 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-99a8568a-52f2-4323-babb-e67aeffd7811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:43.383878 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 689/1357] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:39:43.433393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e390771-182c-4f56-8d94-2f79aab4b505 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:43.439688 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e390771-182c-4f56-8d94-2f79aab4b505 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:43.440075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e390771-182c-4f56-8d94-2f79aab4b505 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:43.443572 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 669/1358] 10.222.0.26 () {54 vars in 961 bytes} [Mon May 9 09:39:43 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:39:43.449881 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4415a-cb61-4b18-a8bc-3aeade3ce757 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:43.450473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4415a-cb61-4b18-a8bc-3aeade3ce757 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:43.450998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4415a-cb61-4b18-a8bc-3aeade3ce757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:43.505428 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-1bd4415a-cb61-4b18-a8bc-3aeade3ce757 None None] MFA Rules not processed for user `6362303cb918486c8525225914332ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:43.506884 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-1bd4415a-cb61-4b18-a8bc-3aeade3ce757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:43.594787 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 690/1359] 10.222.0.26 () {58 vars in 1058 bytes} [Mon May 9 09:39:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:39:43.603101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e390771-182c-4f56-8d94-2f79aab4b505 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:39:43.617731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e390771-182c-4f56-8d94-2f79aab4b505 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:39:43.623298 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3cd82d-40ba-47c2-b965-b5d1aaea8773 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:39:43.624036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cd82d-40ba-47c2-b965-b5d1aaea8773 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:43.624410 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cd82d-40ba-47c2-b965-b5d1aaea8773 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:43.624805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cd82d-40ba-47c2-b965-b5d1aaea8773 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:43.633148 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3cd82d-40ba-47c2-b965-b5d1aaea8773 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:39:43.634760 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3cd82d-40ba-47c2-b965-b5d1aaea8773 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:39:43.644322 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 670/1360] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:39:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:39:46.058540 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016d4c1-9165-4eb4-8200-c3f4b28ed640 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:46.059106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016d4c1-9165-4eb4-8200-c3f4b28ed640 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:46.059421 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016d4c1-9165-4eb4-8200-c3f4b28ed640 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:46.061129 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 691/1361] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:39:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:39:46.066781 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392c817-1916-45ea-9ad0-5c8f5285e5e9 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:46.067255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392c817-1916-45ea-9ad0-5c8f5285e5e9 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:46.067335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392c817-1916-45ea-9ad0-5c8f5285e5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:46.085333 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-8392c817-1916-45ea-9ad0-5c8f5285e5e9 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:46.086530 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-8392c817-1916-45ea-9ad0-5c8f5285e5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:46.094611 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 671/1362] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:39:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:39:46.138917 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016d4c1-9165-4eb4-8200-c3f4b28ed640 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:39:46.153893 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016d4c1-9165-4eb4-8200-c3f4b28ed640 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:39:46.160111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf37032-d13a-4936-8e91-c34c94e633a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:39:46.160806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf37032-d13a-4936-8e91-c34c94e633a8 service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:46.161073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf37032-d13a-4936-8e91-c34c94e633a8 service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:46.161355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf37032-d13a-4936-8e91-c34c94e633a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:46.169851 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf37032-d13a-4936-8e91-c34c94e633a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:39:46.171472 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf37032-d13a-4936-8e91-c34c94e633a8 service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:39:46.181128 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 692/1363] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:39:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:39:48.695333 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266aa94-0f2a-41b1-81a0-599a4f705c0c None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:48.695988 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266aa94-0f2a-41b1-81a0-599a4f705c0c None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:48.695988 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266aa94-0f2a-41b1-81a0-599a4f705c0c None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:48.697271 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 672/1364] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:39:48.702748 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad342ada-1a13-496d-a421-cbfe5408f2c4 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:48.703193 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad342ada-1a13-496d-a421-cbfe5408f2c4 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:48.703291 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad342ada-1a13-496d-a421-cbfe5408f2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:48.730419 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-ad342ada-1a13-496d-a421-cbfe5408f2c4 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:48.740966 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-ad342ada-1a13-496d-a421-cbfe5408f2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:48.762226 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 693/1365] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:39:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:39:48.808146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f87d1c-06d3-421a-a7d2-1e5f96508430 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:48.808426 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f87d1c-06d3-421a-a7d2-1e5f96508430 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:48.808952 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f87d1c-06d3-421a-a7d2-1e5f96508430 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:48.810637 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 673/1366] 10.222.0.26 () {54 vars in 961 bytes} [Mon May 9 09:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:39:48.815500 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cdd46-d5d4-4cff-8b12-f1c17f842f4e None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:48.815894 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cdd46-d5d4-4cff-8b12-f1c17f842f4e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:48.816165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cdd46-d5d4-4cff-8b12-f1c17f842f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:48.834317 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-8d9cdd46-d5d4-4cff-8b12-f1c17f842f4e None None] MFA Rules not processed for user `6362303cb918486c8525225914332ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:48.835761 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-8d9cdd46-d5d4-4cff-8b12-f1c17f842f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:48.846095 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 694/1367] 10.222.0.26 () {58 vars in 1058 bytes} [Mon May 9 09:39:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:39:48.855474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f87d1c-06d3-421a-a7d2-1e5f96508430 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:39:48.870307 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f87d1c-06d3-421a-a7d2-1e5f96508430 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:39:48.877234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7bfde-dbb9-4a96-b376-0ef5d110035c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:39:48.878050 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7bfde-dbb9-4a96-b376-0ef5d110035c service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:48.878328 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7bfde-dbb9-4a96-b376-0ef5d110035c service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:48.878591 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7bfde-dbb9-4a96-b376-0ef5d110035c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:48.887667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7bfde-dbb9-4a96-b376-0ef5d110035c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:39:48.889306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7bfde-dbb9-4a96-b376-0ef5d110035c service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:39:48.898670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 674/1368] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:39:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:39:51.188606 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b572aaf-f972-4497-9dea-a076cc39a79f None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:51.189724 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b572aaf-f972-4497-9dea-a076cc39a79f None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:51.190242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b572aaf-f972-4497-9dea-a076cc39a79f None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:51.192015 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 695/1369] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:39:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:39:51.197826 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94758bed-6091-468b-9abd-77ebb1b2e015 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:51.198323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94758bed-6091-468b-9abd-77ebb1b2e015 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:51.199225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94758bed-6091-468b-9abd-77ebb1b2e015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:51.217945 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-94758bed-6091-468b-9abd-77ebb1b2e015 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:51.219335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-94758bed-6091-468b-9abd-77ebb1b2e015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:51.228411 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 675/1370] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:39:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:39:51.273473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b572aaf-f972-4497-9dea-a076cc39a79f None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:39:51.289335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b572aaf-f972-4497-9dea-a076cc39a79f None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:39:51.296219 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3232da-c82d-410a-93b9-a4d2d2386bef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:39:51.297220 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3232da-c82d-410a-93b9-a4d2d2386bef service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:51.297636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3232da-c82d-410a-93b9-a4d2d2386bef service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:51.298065 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3232da-c82d-410a-93b9-a4d2d2386bef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:51.307764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3232da-c82d-410a-93b9-a4d2d2386bef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:39:51.309739 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3232da-c82d-410a-93b9-a4d2d2386bef service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:39:51.320257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 696/1371] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:39:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:39:51.462438 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bc58c-f3f5-4d9c-bc6c-d0533477583a None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:51.463012 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bc58c-f3f5-4d9c-bc6c-d0533477583a None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:51.463280 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bc58c-f3f5-4d9c-bc6c-d0533477583a None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:51.465035 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 676/1372] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:39:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:39:51.471567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c81a4-f69e-4405-a2a2-f681d9eea98a None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:51.471849 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c81a4-f69e-4405-a2a2-f681d9eea98a None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:51.472122 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c81a4-f69e-4405-a2a2-f681d9eea98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:51.492083 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-c51c81a4-f69e-4405-a2a2-f681d9eea98a None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:51.493399 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c51c81a4-f69e-4405-a2a2-f681d9eea98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:51.501892 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 697/1373] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:39:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:39:51.812533 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a50484-7474-4e6a-aa56-72e285aa3c24 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:51.812964 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a50484-7474-4e6a-aa56-72e285aa3c24 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:51.813538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a50484-7474-4e6a-aa56-72e285aa3c24 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:51.815001 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 677/1374] 10.222.0.26 () {54 vars in 963 bytes} [Mon May 9 09:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:39:51.820290 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4cf6a-ec09-482c-9bff-7d32981ffff9 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:51.820648 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4cf6a-ec09-482c-9bff-7d32981ffff9 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:51.821059 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4cf6a-ec09-482c-9bff-7d32981ffff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:51.871218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-2de4cf6a-ec09-482c-9bff-7d32981ffff9 None None] MFA Rules not processed for user `a44d689993bd4d30858a770012ee3e89`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:51.872418 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-2de4cf6a-ec09-482c-9bff-7d32981ffff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:51.938502 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 698/1375] 10.222.0.26 () {58 vars in 1060 bytes} [Mon May 9 09:39:51 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:39:51.943217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a50484-7474-4e6a-aa56-72e285aa3c24 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:39:51.954953 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a50484-7474-4e6a-aa56-72e285aa3c24 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:39:51.958988 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1135045-e379-4ea1-b8d2-80618f3fcc1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:39:51.960231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1135045-e379-4ea1-b8d2-80618f3fcc1a service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:51.960742 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1135045-e379-4ea1-b8d2-80618f3fcc1a service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:51.961126 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1135045-e379-4ea1-b8d2-80618f3fcc1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:51.970974 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1135045-e379-4ea1-b8d2-80618f3fcc1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:39:51.972544 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1135045-e379-4ea1-b8d2-80618f3fcc1a service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:39:51.982937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 678/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:39:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:39:53.583111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087d5ed-1abc-422a-aa06-03b4fc29b5bb None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:53.583767 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087d5ed-1abc-422a-aa06-03b4fc29b5bb None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:53.583767 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087d5ed-1abc-422a-aa06-03b4fc29b5bb None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:53.585166 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 699/1377] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:39:53.591034 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30c366-9fbf-4b99-852d-237063c6d5b5 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:53.591583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30c366-9fbf-4b99-852d-237063c6d5b5 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:53.591583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30c366-9fbf-4b99-852d-237063c6d5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:53.610918 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-3b30c366-9fbf-4b99-852d-237063c6d5b5 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:53.612178 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-3b30c366-9fbf-4b99-852d-237063c6d5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:53.620997 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 679/1378] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:39:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:39:53.665492 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087d5ed-1abc-422a-aa06-03b4fc29b5bb None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:39:53.680976 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087d5ed-1abc-422a-aa06-03b4fc29b5bb None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:39:53.687082 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ed5b7-e793-45a7-871a-f3cbf6e90753 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:39:53.687733 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ed5b7-e793-45a7-871a-f3cbf6e90753 service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:53.688000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ed5b7-e793-45a7-871a-f3cbf6e90753 service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:53.688303 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ed5b7-e793-45a7-871a-f3cbf6e90753 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:53.697559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ed5b7-e793-45a7-871a-f3cbf6e90753 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:39:53.699180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ed5b7-e793-45a7-871a-f3cbf6e90753 service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:39:53.709060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 700/1379] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:39:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:39:55.949603 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefbc31-1684-4705-830b-1c2968eff91e None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:55.950335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefbc31-1684-4705-830b-1c2968eff91e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:55.950335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefbc31-1684-4705-830b-1c2968eff91e None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:55.953019 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 680/1380] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:39:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:39:55.956640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f456ae0-0516-43a5-b853-5ec76f8b5c53 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:55.957132 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f456ae0-0516-43a5-b853-5ec76f8b5c53 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:55.957250 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f456ae0-0516-43a5-b853-5ec76f8b5c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:55.975259 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-9f456ae0-0516-43a5-b853-5ec76f8b5c53 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:55.976507 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-9f456ae0-0516-43a5-b853-5ec76f8b5c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:55.984916 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 701/1381] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:39:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:39:56.027806 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefbc31-1684-4705-830b-1c2968eff91e None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:39:56.043537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefbc31-1684-4705-830b-1c2968eff91e None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:39:56.049052 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8dd74-9fdc-4626-8794-d784d4526057 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:39:56.049695 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8dd74-9fdc-4626-8794-d784d4526057 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:56.049968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8dd74-9fdc-4626-8794-d784d4526057 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:56.050232 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8dd74-9fdc-4626-8794-d784d4526057 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:56.058865 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8dd74-9fdc-4626-8794-d784d4526057 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:39:56.060440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8dd74-9fdc-4626-8794-d784d4526057 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:39:56.070407 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 681/1382] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:39:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:39:58.299399 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36821391-0af9-462a-890d-cbd270bfe58b None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:58.300001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36821391-0af9-462a-890d-cbd270bfe58b None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:58.300001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36821391-0af9-462a-890d-cbd270bfe58b None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:58.301302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 702/1383] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:39:58.307105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fef6e-1616-4451-b5ab-ed1bb8a7443e None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:58.307656 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fef6e-1616-4451-b5ab-ed1bb8a7443e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:58.307656 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fef6e-1616-4451-b5ab-ed1bb8a7443e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:58.328070 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-9d3fef6e-1616-4451-b5ab-ed1bb8a7443e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:39:58.329620 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-9d3fef6e-1616-4451-b5ab-ed1bb8a7443e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:39:58.339423 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 682/1384] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:39:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:39:58.382386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36821391-0af9-462a-890d-cbd270bfe58b None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:39:58.397175 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36821391-0af9-462a-890d-cbd270bfe58b None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:39:58.402900 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc527cc-be5d-40eb-9069-78f2f833920d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:39:58.403544 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc527cc-be5d-40eb-9069-78f2f833920d service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:39:58.403819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc527cc-be5d-40eb-9069-78f2f833920d service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:39:58.404085 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc527cc-be5d-40eb-9069-78f2f833920d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:39:58.412655 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc527cc-be5d-40eb-9069-78f2f833920d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:39:58.414327 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc527cc-be5d-40eb-9069-78f2f833920d service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:39:58.424510 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 703/1385] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:39:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:40:00.607246 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69967684-5555-49da-b614-ebb7c6f026ea None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:00.607867 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69967684-5555-49da-b614-ebb7c6f026ea None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:00.607867 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69967684-5555-49da-b614-ebb7c6f026ea None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:00.609198 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 683/1386] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:00.616438 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73b62b-bab8-442d-b364-f0bba1b9ec55 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:00.616919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73b62b-bab8-442d-b364-f0bba1b9ec55 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:00.617143 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73b62b-bab8-442d-b364-f0bba1b9ec55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:00.642398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-7c73b62b-bab8-442d-b364-f0bba1b9ec55 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:00.643606 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-7c73b62b-bab8-442d-b364-f0bba1b9ec55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:00.653778 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 704/1387] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:40:00.700231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69967684-5555-49da-b614-ebb7c6f026ea None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:40:00.716001 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69967684-5555-49da-b614-ebb7c6f026ea None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:40:00.721962 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c822a6-fe2b-4848-af70-58550433078c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:40:00.722600 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c822a6-fe2b-4848-af70-58550433078c service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:00.722878 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c822a6-fe2b-4848-af70-58550433078c service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:00.723151 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c822a6-fe2b-4848-af70-58550433078c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:00.731958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c822a6-fe2b-4848-af70-58550433078c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:40:00.733679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c822a6-fe2b-4848-af70-58550433078c service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:40:00.743868 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 684/1388] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:40:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:40:02.992280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d20990-729e-47f8-be58-bee305f52f52 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:02.992855 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d20990-729e-47f8-be58-bee305f52f52 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:02.992855 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d20990-729e-47f8-be58-bee305f52f52 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:02.994205 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 705/1389] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:03.000038 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4b741-e86e-4ee1-b9c6-d6d1511dc1f4 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:03.000505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4b741-e86e-4ee1-b9c6-d6d1511dc1f4 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:03.000750 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4b741-e86e-4ee1-b9c6-d6d1511dc1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:03.021535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-c7f4b741-e86e-4ee1-b9c6-d6d1511dc1f4 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:03.022829 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-c7f4b741-e86e-4ee1-b9c6-d6d1511dc1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:03.031448 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 685/1390] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:40:03.075781 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d20990-729e-47f8-be58-bee305f52f52 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:40:03.090102 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d20990-729e-47f8-be58-bee305f52f52 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:40:03.095828 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7548ce7-e634-41c0-b662-f667d32ebcab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:40:03.096465 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7548ce7-e634-41c0-b662-f667d32ebcab service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:03.096772 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7548ce7-e634-41c0-b662-f667d32ebcab service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:03.097041 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7548ce7-e634-41c0-b662-f667d32ebcab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:03.104891 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7548ce7-e634-41c0-b662-f667d32ebcab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:40:03.106514 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7548ce7-e634-41c0-b662-f667d32ebcab service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:40:03.116054 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 706/1391] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:40:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 09:40:05.408196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417479a9-c4f9-4ebd-985a-b68e3c251e71 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:05.408825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417479a9-c4f9-4ebd-985a-b68e3c251e71 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:05.408825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417479a9-c4f9-4ebd-985a-b68e3c251e71 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:05.410157 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 686/1392] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:05.415540 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c4781-c753-48ae-8df1-d2405a3161ab None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:05.416116 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c4781-c753-48ae-8df1-d2405a3161ab None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:05.416116 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c4781-c753-48ae-8df1-d2405a3161ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:05.434052 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-3c5c4781-c753-48ae-8df1-d2405a3161ab None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:05.435282 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-3c5c4781-c753-48ae-8df1-d2405a3161ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:05.443106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 707/1393] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:40:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:40:05.489541 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417479a9-c4f9-4ebd-985a-b68e3c251e71 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:40:05.504098 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417479a9-c4f9-4ebd-985a-b68e3c251e71 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:40:05.509640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19955f87-8826-4159-bcc3-d7969a8cdab4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:40:05.510268 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19955f87-8826-4159-bcc3-d7969a8cdab4 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:05.510547 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19955f87-8826-4159-bcc3-d7969a8cdab4 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:05.510820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19955f87-8826-4159-bcc3-d7969a8cdab4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:05.519786 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19955f87-8826-4159-bcc3-d7969a8cdab4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:40:05.521899 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19955f87-8826-4159-bcc3-d7969a8cdab4 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:40:05.534459 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 687/1394] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:40:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:40:07.774430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f333d7-448d-4e57-8572-ea701f98400e None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:07.774967 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f333d7-448d-4e57-8572-ea701f98400e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:07.774967 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f333d7-448d-4e57-8572-ea701f98400e None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:07.776440 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 708/1395] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:07.782450 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfd74c-136e-481e-a147-4c5f923981df None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:07.782923 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfd74c-136e-481e-a147-4c5f923981df None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:07.783192 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfd74c-136e-481e-a147-4c5f923981df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:07.805942 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-e2bfd74c-136e-481e-a147-4c5f923981df None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:07.807825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e2bfd74c-136e-481e-a147-4c5f923981df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:07.818513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 688/1396] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:40:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:40:07.864935 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f333d7-448d-4e57-8572-ea701f98400e None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:40:07.881082 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f333d7-448d-4e57-8572-ea701f98400e None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:40:07.886728 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd4ab9-3625-4c42-b869-4b90b292ae38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:40:07.887335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd4ab9-3625-4c42-b869-4b90b292ae38 service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:07.887614 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd4ab9-3625-4c42-b869-4b90b292ae38 service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:07.887883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd4ab9-3625-4c42-b869-4b90b292ae38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:07.896303 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd4ab9-3625-4c42-b869-4b90b292ae38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:40:07.898021 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd4ab9-3625-4c42-b869-4b90b292ae38 service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:40:07.907375 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 709/1397] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:40:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:40:10.162280 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941c8c7-38c4-4500-a7f7-9ef584235587 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:10.162927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941c8c7-38c4-4500-a7f7-9ef584235587 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:10.162927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941c8c7-38c4-4500-a7f7-9ef584235587 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:10.164242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 689/1398] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:10.169837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c4584-24b3-4748-8e65-67185e0a7cee None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:10.170308 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c4584-24b3-4748-8e65-67185e0a7cee None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:10.170418 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c4584-24b3-4748-8e65-67185e0a7cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:10.188573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-9e9c4584-24b3-4748-8e65-67185e0a7cee None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:10.189848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-9e9c4584-24b3-4748-8e65-67185e0a7cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:10.198764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 710/1399] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:40:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:40:10.242931 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941c8c7-38c4-4500-a7f7-9ef584235587 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:40:10.258514 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941c8c7-38c4-4500-a7f7-9ef584235587 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:40:10.264312 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e51529-8521-45c9-b65e-b3ae633776f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:40:10.264975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e51529-8521-45c9-b65e-b3ae633776f2 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:10.265259 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e51529-8521-45c9-b65e-b3ae633776f2 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:10.265513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e51529-8521-45c9-b65e-b3ae633776f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:10.273831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e51529-8521-45c9-b65e-b3ae633776f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:40:10.275413 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e51529-8521-45c9-b65e-b3ae633776f2 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:40:10.284618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 690/1400] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:40:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:40:10.702013 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b6f95-2729-4968-a33f-333e2ff618e2 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:10.702400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b6f95-2729-4968-a33f-333e2ff618e2 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:10.702858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b6f95-2729-4968-a33f-333e2ff618e2 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:10.705126 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 711/1401] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:40:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:10.710576 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2c978-a57f-408d-a49b-26144212a919 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:10.710894 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2c978-a57f-408d-a49b-26144212a919 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:10.711166 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2c978-a57f-408d-a49b-26144212a919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:10.730284 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-58e2c978-a57f-408d-a49b-26144212a919 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:10.731474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-58e2c978-a57f-408d-a49b-26144212a919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:10.740934 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 691/1402] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:40:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:40:10.799046 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc33ff9-15d9-4d81-a961-7d1daa00b4fa None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:10.799329 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc33ff9-15d9-4d81-a961-7d1daa00b4fa None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:10.799775 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc33ff9-15d9-4d81-a961-7d1daa00b4fa None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:10.801711 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 712/1403] 10.222.0.26 () {54 vars in 962 bytes} [Mon May 9 09:40:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:10.807114 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda013c-94aa-4d64-8fa1-80c1a3d7a48d None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:10.807467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda013c-94aa-4d64-8fa1-80c1a3d7a48d None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:10.807745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda013c-94aa-4d64-8fa1-80c1a3d7a48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:10.861717 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-bfda013c-94aa-4d64-8fa1-80c1a3d7a48d None None] MFA Rules not processed for user `7fd398c547554629927d2a80b3a8dee6`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:10.862834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-bfda013c-94aa-4d64-8fa1-80c1a3d7a48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:10.927225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 692/1404] 10.222.0.26 () {58 vars in 1059 bytes} [Mon May 9 09:40:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:40:10.938903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc33ff9-15d9-4d81-a961-7d1daa00b4fa None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:40:10.953053 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc33ff9-15d9-4d81-a961-7d1daa00b4fa None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:40:10.957563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7df6e-4a97-4c09-91e7-b740c45b06fe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fd398c547554629927d2a80b3a8dee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:40:10.958225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7df6e-4a97-4c09-91e7-b740c45b06fe service glance] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:10.958501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7df6e-4a97-4c09-91e7-b740c45b06fe service glance] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:10.958771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7df6e-4a97-4c09-91e7-b740c45b06fe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:10.968158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7df6e-4a97-4c09-91e7-b740c45b06fe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:40:10.969629 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7df6e-4a97-4c09-91e7-b740c45b06fe service glance] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:40:10.979030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 713/1405] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:40:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:40:11.410533 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41127aa0-0247-4700-ab54-25aceadcb43d None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:11.411066 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41127aa0-0247-4700-ab54-25aceadcb43d None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:11.411066 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41127aa0-0247-4700-ab54-25aceadcb43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:11.458315 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-41127aa0-0247-4700-ab54-25aceadcb43d None None] MFA Rules not processed for user `af62b2d3f73f482b8db4173af33752c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:11.459406 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-41127aa0-0247-4700-ab54-25aceadcb43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:11.532816 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 693/1406] 10.222.0.26 () {58 vars in 1009 bytes} [Mon May 9 09:40:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:40:11.577503 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e0621-39db-4739-ba4a-787de9a684b8 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:11.578002 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e0621-39db-4739-ba4a-787de9a684b8 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:11.578002 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e0621-39db-4739-ba4a-787de9a684b8 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:11.579437 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 714/1407] 10.222.0.26 () {54 vars in 942 bytes} [Mon May 9 09:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:11.585730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a7086-5ba7-4b44-9841-a3b532599e40 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:11.586008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a7086-5ba7-4b44-9841-a3b532599e40 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:11.586280 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a7086-5ba7-4b44-9841-a3b532599e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:11.646544 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-a93a7086-5ba7-4b44-9841-a3b532599e40 None None] MFA Rules not processed for user `380e04358f2349aa912d6a97518877cd`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:11.648105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a93a7086-5ba7-4b44-9841-a3b532599e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:11.746905 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 694/1408] 10.222.0.26 () {58 vars in 1039 bytes} [Mon May 9 09:40:11 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:40:11.756337 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e0621-39db-4739-ba4a-787de9a684b8 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:40:11.771638 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e0621-39db-4739-ba4a-787de9a684b8 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:40:11.777633 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab37017-0eb4-4877-8645-d853be7fb02e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380e04358f2349aa912d6a97518877cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:40:11.778278 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab37017-0eb4-4877-8645-d853be7fb02e service swift] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:11.778559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab37017-0eb4-4877-8645-d853be7fb02e service swift] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:11.778824 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab37017-0eb4-4877-8645-d853be7fb02e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:11.787626 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab37017-0eb4-4877-8645-d853be7fb02e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:40:11.789345 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab37017-0eb4-4877-8645-d853be7fb02e service swift] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:40:11.797568 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 715/1409] 10.222.0.26 () {60 vars in 1384 bytes} [Mon May 9 09:40:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 09:40:35.200884 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15005149-8746-4568-a0de-0590df514379 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:35.201592 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15005149-8746-4568-a0de-0590df514379 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:35.201592 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15005149-8746-4568-a0de-0590df514379 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:35.202813 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 695/1410] 10.222.0.26 () {54 vars in 918 bytes} [Mon May 9 09:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:40:35.208185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eba49a-de7f-4097-a40b-38964b13df98 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:35.208684 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eba49a-de7f-4097-a40b-38964b13df98 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:35.208808 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eba49a-de7f-4097-a40b-38964b13df98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:35.227411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-24eba49a-de7f-4097-a40b-38964b13df98 None None] MFA Rules not processed for user `6c9e458c2c5f4ed1b3eddd84c2ec1703`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:35.228627 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-24eba49a-de7f-4097-a40b-38964b13df98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:35.239222 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 716/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Mon May 9 09:40:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:40:35.253494 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac25b08-9643-4a9f-b082-352734a0f165 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:35.253774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac25b08-9643-4a9f-b082-352734a0f165 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:35.254047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac25b08-9643-4a9f-b082-352734a0f165 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:35.255393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 696/1412] 10.222.0.26 () {54 vars in 918 bytes} [Mon May 9 09:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:35.260064 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064840ff-9746-488e-ae4a-18f8c14e63f0 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:35.260347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064840ff-9746-488e-ae4a-18f8c14e63f0 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:35.260616 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064840ff-9746-488e-ae4a-18f8c14e63f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:35.280020 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-064840ff-9746-488e-ae4a-18f8c14e63f0 None None] MFA Rules not processed for user `6362303cb918486c8525225914332ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:35.281231 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-064840ff-9746-488e-ae4a-18f8c14e63f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:35.292074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 717/1413] 10.222.0.26 () {58 vars in 1015 bytes} [Mon May 9 09:40:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:40:37.950114 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429560b1-b98b-4acf-b57a-1d1a193ba253 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:37.950785 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429560b1-b98b-4acf-b57a-1d1a193ba253 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:37.950785 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429560b1-b98b-4acf-b57a-1d1a193ba253 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:37.952593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 697/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 09:40:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:37.958175 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3b4f5-6536-47aa-8dd7-2b9e0a3b252d None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:37.958609 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3b4f5-6536-47aa-8dd7-2b9e0a3b252d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:37.958723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3b4f5-6536-47aa-8dd7-2b9e0a3b252d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:37.979752 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-d5e3b4f5-6536-47aa-8dd7-2b9e0a3b252d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:37.980992 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-d5e3b4f5-6536-47aa-8dd7-2b9e0a3b252d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:37.990273 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 718/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 09:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:40:38.032929 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429560b1-b98b-4acf-b57a-1d1a193ba253 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:40:38.051628 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429560b1-b98b-4acf-b57a-1d1a193ba253 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:40:38.058624 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0a4c66-ed62-4a96-bafa-d88653963319 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:40:38.059261 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a4c66-ed62-4a96-bafa-d88653963319 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:38.059537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a4c66-ed62-4a96-bafa-d88653963319 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:38.059807 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a4c66-ed62-4a96-bafa-d88653963319 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:38.069172 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0a4c66-ed62-4a96-bafa-d88653963319 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:40:38.070853 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0a4c66-ed62-4a96-bafa-d88653963319 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:40:38.081017 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 698/1416] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:40:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:40:39.082346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab8aa9-b19c-4f8f-97a2-7575594fab01 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:39.082978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab8aa9-b19c-4f8f-97a2-7575594fab01 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:39.082978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab8aa9-b19c-4f8f-97a2-7575594fab01 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:39.084333 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 719/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 09:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:39.090680 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b629ac-2709-45d9-9a28-cde217b6ab6d None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:39.091137 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b629ac-2709-45d9-9a28-cde217b6ab6d None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:39.091257 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b629ac-2709-45d9-9a28-cde217b6ab6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:39.119030 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-07b629ac-2709-45d9-9a28-cde217b6ab6d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:39.120283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-07b629ac-2709-45d9-9a28-cde217b6ab6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:39.130275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 699/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 09:40:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:40:39.303176 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab8aa9-b19c-4f8f-97a2-7575594fab01 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:40:39.318611 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab8aa9-b19c-4f8f-97a2-7575594fab01 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:40:39.324043 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000d347-a5ab-45f1-873e-a440fda4c9bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:40:39.324696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000d347-a5ab-45f1-873e-a440fda4c9bb service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:39.324981 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000d347-a5ab-45f1-873e-a440fda4c9bb service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:39.325250 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000d347-a5ab-45f1-873e-a440fda4c9bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:39.333743 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000d347-a5ab-45f1-873e-a440fda4c9bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:40:39.335246 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000d347-a5ab-45f1-873e-a440fda4c9bb service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:40:39.344975 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 720/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:40:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:40:40.339885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd78c82-1777-4c49-a274-28ab43c70c07 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:40.340465 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd78c82-1777-4c49-a274-28ab43c70c07 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:40.340465 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd78c82-1777-4c49-a274-28ab43c70c07 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:40.341855 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 700/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 09:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:40.350154 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdf40d-0ba4-43e7-a026-d184905edd6b None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:40.350949 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdf40d-0ba4-43e7-a026-d184905edd6b None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:40.351111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdf40d-0ba4-43e7-a026-d184905edd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:40.373495 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-6bfdf40d-0ba4-43e7-a026-d184905edd6b None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:40.374661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-6bfdf40d-0ba4-43e7-a026-d184905edd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:40.382949 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 721/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 09:40:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:40:40.536908 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd78c82-1777-4c49-a274-28ab43c70c07 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:40:40.553777 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd78c82-1777-4c49-a274-28ab43c70c07 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:40:40.560115 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e1442-4159-434f-ae8c-2ba8ac7ddf22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:40:40.560611 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e1442-4159-434f-ae8c-2ba8ac7ddf22 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:40.560918 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e1442-4159-434f-ae8c-2ba8ac7ddf22 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:40.561190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e1442-4159-434f-ae8c-2ba8ac7ddf22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:40.569766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e1442-4159-434f-ae8c-2ba8ac7ddf22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:40:40.571451 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e1442-4159-434f-ae8c-2ba8ac7ddf22 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:40:40.581238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 701/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:40:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:40:41.491527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9256d0-2870-4fc0-9f66-d79cf1f8857f None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:41.492022 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9256d0-2870-4fc0-9f66-d79cf1f8857f None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:41.492119 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9256d0-2870-4fc0-9f66-d79cf1f8857f None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:41.494296 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 722/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 09:40:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:41.500217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff77393-08d3-4dc1-a2b1-0fb0444069a1 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:41.500696 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff77393-08d3-4dc1-a2b1-0fb0444069a1 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:41.500815 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff77393-08d3-4dc1-a2b1-0fb0444069a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:41.521921 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-8ff77393-08d3-4dc1-a2b1-0fb0444069a1 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:41.523123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-8ff77393-08d3-4dc1-a2b1-0fb0444069a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:41.532502 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 702/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 09:40:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:40:41.682868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9256d0-2870-4fc0-9f66-d79cf1f8857f None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:40:41.698707 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9256d0-2870-4fc0-9f66-d79cf1f8857f None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:40:41.704379 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb7603-1e9d-4bf6-b496-40fa898007c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:40:41.705041 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb7603-1e9d-4bf6-b496-40fa898007c0 service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:41.705319 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb7603-1e9d-4bf6-b496-40fa898007c0 service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:41.705582 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb7603-1e9d-4bf6-b496-40fa898007c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:41.713835 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb7603-1e9d-4bf6-b496-40fa898007c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:40:41.715416 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb7603-1e9d-4bf6-b496-40fa898007c0 service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:40:41.724473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 723/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:40:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:40:42.645320 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509258c-3f5c-445b-9ffb-66f891970832 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:42.645968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509258c-3f5c-445b-9ffb-66f891970832 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:42.645968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509258c-3f5c-445b-9ffb-66f891970832 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:42.647221 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 703/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 09:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:42.653709 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b569dc3-cee0-4bed-8d91-2d5e8c700390 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:42.654460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b569dc3-cee0-4bed-8d91-2d5e8c700390 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:42.654623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b569dc3-cee0-4bed-8d91-2d5e8c700390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:42.678752 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-5b569dc3-cee0-4bed-8d91-2d5e8c700390 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:42.680325 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-5b569dc3-cee0-4bed-8d91-2d5e8c700390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:42.689603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 724/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 09:40:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:40:42.838573 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509258c-3f5c-445b-9ffb-66f891970832 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:40:42.860129 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509258c-3f5c-445b-9ffb-66f891970832 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:40:42.866798 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9df2fe-4d85-44f6-91c8-d9598a8ce618 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:40:42.867447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9df2fe-4d85-44f6-91c8-d9598a8ce618 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:42.867743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9df2fe-4d85-44f6-91c8-d9598a8ce618 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:42.868013 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9df2fe-4d85-44f6-91c8-d9598a8ce618 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:42.877394 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9df2fe-4d85-44f6-91c8-d9598a8ce618 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:40:42.878991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9df2fe-4d85-44f6-91c8-d9598a8ce618 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:40:42.888529 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 704/1428] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:40:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:40:43.745183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1324d3f-a360-468f-9454-0c9577c15851 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:43.745747 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1324d3f-a360-468f-9454-0c9577c15851 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:43.745747 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1324d3f-a360-468f-9454-0c9577c15851 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:43.747008 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 725/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 09:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:43.752057 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064c3b8-e06f-4937-a29c-ac4a1cc8aa6a None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:43.752512 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064c3b8-e06f-4937-a29c-ac4a1cc8aa6a None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:43.752608 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064c3b8-e06f-4937-a29c-ac4a1cc8aa6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:43.772156 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-7064c3b8-e06f-4937-a29c-ac4a1cc8aa6a None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:43.773342 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-7064c3b8-e06f-4937-a29c-ac4a1cc8aa6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:43.784589 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 705/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 09:40:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:40:43.938450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1324d3f-a360-468f-9454-0c9577c15851 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:40:43.962359 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1324d3f-a360-468f-9454-0c9577c15851 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:40:43.968940 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf1dcb-ab33-414b-aed8-72da694f969e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:40:43.969577 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf1dcb-ab33-414b-aed8-72da694f969e service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:43.969852 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf1dcb-ab33-414b-aed8-72da694f969e service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:43.970118 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf1dcb-ab33-414b-aed8-72da694f969e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:43.978645 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf1dcb-ab33-414b-aed8-72da694f969e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:40:43.980220 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf1dcb-ab33-414b-aed8-72da694f969e service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:40:43.990329 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 726/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:40:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:40:44.865121 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731babd3-b6fa-45a9-a5da-ea7bda91f0e5 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:44.865996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731babd3-b6fa-45a9-a5da-ea7bda91f0e5 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:44.865996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731babd3-b6fa-45a9-a5da-ea7bda91f0e5 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:44.868238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 706/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 09:40:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:44.875689 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b53043-5fcd-44ba-9f2d-882d1f75ec1c None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:44.876167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b53043-5fcd-44ba-9f2d-882d1f75ec1c None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:44.876273 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b53043-5fcd-44ba-9f2d-882d1f75ec1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:44.895065 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-08b53043-5fcd-44ba-9f2d-882d1f75ec1c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:44.896293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-08b53043-5fcd-44ba-9f2d-882d1f75ec1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:44.904640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 727/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 09:40:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:40:45.055411 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731babd3-b6fa-45a9-a5da-ea7bda91f0e5 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:40:45.078489 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731babd3-b6fa-45a9-a5da-ea7bda91f0e5 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:40:45.085819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349bf0d6-86f7-4c36-afe9-5bb506e48cb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:40:45.086540 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bf0d6-86f7-4c36-afe9-5bb506e48cb2 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:45.086825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bf0d6-86f7-4c36-afe9-5bb506e48cb2 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:45.087070 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bf0d6-86f7-4c36-afe9-5bb506e48cb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:45.095844 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349bf0d6-86f7-4c36-afe9-5bb506e48cb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:40:45.097513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349bf0d6-86f7-4c36-afe9-5bb506e48cb2 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:40:45.107093 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 707/1434] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:40:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:40:45.985162 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddce87f-e3c7-4174-a1d2-0b0700cd6809 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:45.986178 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddce87f-e3c7-4174-a1d2-0b0700cd6809 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:45.986178 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddce87f-e3c7-4174-a1d2-0b0700cd6809 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:45.988712 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 728/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 09:40:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:45.994143 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d5f8c-52f4-4de9-ba2b-519d52e96f9e None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:45.994585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d5f8c-52f4-4de9-ba2b-519d52e96f9e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:45.994704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d5f8c-52f4-4de9-ba2b-519d52e96f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:46.013934 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-cd6d5f8c-52f4-4de9-ba2b-519d52e96f9e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:46.015138 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-cd6d5f8c-52f4-4de9-ba2b-519d52e96f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:46.024475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 708/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 09:40:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:40:46.171607 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddce87f-e3c7-4174-a1d2-0b0700cd6809 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:40:46.188264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddce87f-e3c7-4174-a1d2-0b0700cd6809 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:40:46.194490 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc78223-0e83-415f-84f6-21cdf3d9510a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:40:46.195149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc78223-0e83-415f-84f6-21cdf3d9510a service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:46.195406 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc78223-0e83-415f-84f6-21cdf3d9510a service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:46.195715 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc78223-0e83-415f-84f6-21cdf3d9510a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:46.207031 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc78223-0e83-415f-84f6-21cdf3d9510a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:40:46.209217 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc78223-0e83-415f-84f6-21cdf3d9510a service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:40:46.223729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 729/1437] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:40:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:40:47.094134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739cfe8-dcab-447d-a679-a157a12a488b None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:47.095063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739cfe8-dcab-447d-a679-a157a12a488b None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:47.095063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739cfe8-dcab-447d-a679-a157a12a488b None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:47.097346 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 709/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 09:40:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:47.102528 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d480c-05a3-49a3-8c74-b18a7e746b7c None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:47.102960 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d480c-05a3-49a3-8c74-b18a7e746b7c None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:47.103055 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d480c-05a3-49a3-8c74-b18a7e746b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:47.121557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-786d480c-05a3-49a3-8c74-b18a7e746b7c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:47.122742 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-786d480c-05a3-49a3-8c74-b18a7e746b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:47.131010 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 730/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 09:40:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:40:47.281446 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739cfe8-dcab-447d-a679-a157a12a488b None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:40:47.301554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739cfe8-dcab-447d-a679-a157a12a488b None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:40:47.309790 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41666f-bbf9-420a-a5a4-74dd2bc17591 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:40:47.310421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41666f-bbf9-420a-a5a4-74dd2bc17591 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:47.310693 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41666f-bbf9-420a-a5a4-74dd2bc17591 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:47.310958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41666f-bbf9-420a-a5a4-74dd2bc17591 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:47.319659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e41666f-bbf9-420a-a5a4-74dd2bc17591 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:40:47.321289 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e41666f-bbf9-420a-a5a4-74dd2bc17591 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:40:47.330933 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 710/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:40:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:40:49.418557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9882b0-5f84-4ce1-a485-d27f159ca991 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:49.419144 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9882b0-5f84-4ce1-a485-d27f159ca991 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:49.419144 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9882b0-5f84-4ce1-a485-d27f159ca991 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:49.420533 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 731/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 09:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:49.426139 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578b7bf-0f0d-4fcd-af62-81d2345ff03e None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:49.426949 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578b7bf-0f0d-4fcd-af62-81d2345ff03e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:49.427225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578b7bf-0f0d-4fcd-af62-81d2345ff03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:49.444843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-c578b7bf-0f0d-4fcd-af62-81d2345ff03e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:49.446146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-c578b7bf-0f0d-4fcd-af62-81d2345ff03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:49.454418 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 711/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 09:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:40:49.492545 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9882b0-5f84-4ce1-a485-d27f159ca991 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:40:49.503703 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9882b0-5f84-4ce1-a485-d27f159ca991 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:40:49.507457 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10fc704-8ceb-4a77-824c-25a70a736036 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:40:49.508182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fc704-8ceb-4a77-824c-25a70a736036 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:49.508447 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fc704-8ceb-4a77-824c-25a70a736036 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:49.508744 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fc704-8ceb-4a77-824c-25a70a736036 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:49.518806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10fc704-8ceb-4a77-824c-25a70a736036 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:40:49.520329 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10fc704-8ceb-4a77-824c-25a70a736036 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:40:49.530768 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 732/1443] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:40:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:40:51.856534 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd35add-8070-47f8-86ae-9acff0d53faa None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:51.857183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd35add-8070-47f8-86ae-9acff0d53faa None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:51.857183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd35add-8070-47f8-86ae-9acff0d53faa None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:51.858508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 712/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 09:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:51.865263 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265658e-1da9-4409-b675-0de3ecc69759 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:51.866007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265658e-1da9-4409-b675-0de3ecc69759 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:51.866204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265658e-1da9-4409-b675-0de3ecc69759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:51.885526 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-e265658e-1da9-4409-b675-0de3ecc69759 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:51.886736 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e265658e-1da9-4409-b675-0de3ecc69759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:51.895728 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 733/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 09:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:40:51.940051 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd35add-8070-47f8-86ae-9acff0d53faa None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:40:51.957533 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd35add-8070-47f8-86ae-9acff0d53faa None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:40:51.962827 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d8824-d64c-47fb-acd3-2e9227b0f0e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:40:51.963454 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d8824-d64c-47fb-acd3-2e9227b0f0e3 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:51.963727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d8824-d64c-47fb-acd3-2e9227b0f0e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:51.963996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d8824-d64c-47fb-acd3-2e9227b0f0e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:51.973037 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d8824-d64c-47fb-acd3-2e9227b0f0e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:40:51.974465 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d8824-d64c-47fb-acd3-2e9227b0f0e3 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:40:51.983654 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 713/1446] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:40:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:40:54.492863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71a557-d7ea-426b-a6a9-1ac069414175 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:54.493473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71a557-d7ea-426b-a6a9-1ac069414175 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:54.493473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71a557-d7ea-426b-a6a9-1ac069414175 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:54.495010 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 734/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 09:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:54.501310 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1196636-a5c2-4a53-a004-c8f1870333fa None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:54.502026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1196636-a5c2-4a53-a004-c8f1870333fa None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:54.502193 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1196636-a5c2-4a53-a004-c8f1870333fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:54.523418 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-a1196636-a5c2-4a53-a004-c8f1870333fa None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:54.524616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a1196636-a5c2-4a53-a004-c8f1870333fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:54.533597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 714/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 09:40:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:40:54.572757 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71a557-d7ea-426b-a6a9-1ac069414175 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:40:54.584867 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71a557-d7ea-426b-a6a9-1ac069414175 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:40:54.588857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e713c-c7c9-4729-b0c1-6e5c1d432912 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:40:54.589493 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e713c-c7c9-4729-b0c1-6e5c1d432912 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:54.589768 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e713c-c7c9-4729-b0c1-6e5c1d432912 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:54.590060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e713c-c7c9-4729-b0c1-6e5c1d432912 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:54.601755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e713c-c7c9-4729-b0c1-6e5c1d432912 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:40:54.605647 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e713c-c7c9-4729-b0c1-6e5c1d432912 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:40:54.620922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 735/1449] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:40:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:40:56.909093 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc13cc-0434-4945-a0c0-b4483c5219df None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:56.909689 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc13cc-0434-4945-a0c0-b4483c5219df None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:56.909689 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc13cc-0434-4945-a0c0-b4483c5219df None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:56.910929 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 715/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 09:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:56.917968 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb8e4d-5e76-4264-a7d1-9472adc8fdf4 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:56.918432 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb8e4d-5e76-4264-a7d1-9472adc8fdf4 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:56.918537 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb8e4d-5e76-4264-a7d1-9472adc8fdf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:56.938673 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-3bdb8e4d-5e76-4264-a7d1-9472adc8fdf4 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:56.939869 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-3bdb8e4d-5e76-4264-a7d1-9472adc8fdf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:56.949806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 736/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 09:40:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:40:56.990698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc13cc-0434-4945-a0c0-b4483c5219df None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:40:57.009703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc13cc-0434-4945-a0c0-b4483c5219df None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:40:57.014457 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23707861-f265-48c4-bede-f0a496cbf626 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:40:57.015103 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23707861-f265-48c4-bede-f0a496cbf626 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:57.015380 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23707861-f265-48c4-bede-f0a496cbf626 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:57.015648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23707861-f265-48c4-bede-f0a496cbf626 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:57.025245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23707861-f265-48c4-bede-f0a496cbf626 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:40:57.026685 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23707861-f265-48c4-bede-f0a496cbf626 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:40:57.035825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 716/1452] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:40:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:40:59.351960 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a428c57-1637-4364-b92c-cb4a35950fd9 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:59.352910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a428c57-1637-4364-b92c-cb4a35950fd9 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:59.352910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a428c57-1637-4364-b92c-cb4a35950fd9 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:59.355125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 737/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 09:40:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:40:59.364170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1b89a-a52b-4853-ae3e-87fae6d9bb43 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:59.365006 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1b89a-a52b-4853-ae3e-87fae6d9bb43 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:59.365184 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1b89a-a52b-4853-ae3e-87fae6d9bb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:59.384834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-8df1b89a-a52b-4853-ae3e-87fae6d9bb43 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:40:59.386027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-8df1b89a-a52b-4853-ae3e-87fae6d9bb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:40:59.394478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 717/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 09:40:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:40:59.431302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a428c57-1637-4364-b92c-cb4a35950fd9 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:40:59.449692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a428c57-1637-4364-b92c-cb4a35950fd9 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:40:59.455264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dfe5f-2995-4300-ac96-e7443b43403f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:40:59.455891 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dfe5f-2995-4300-ac96-e7443b43403f service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:40:59.456167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dfe5f-2995-4300-ac96-e7443b43403f service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:40:59.456426 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dfe5f-2995-4300-ac96-e7443b43403f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:40:59.465986 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63dfe5f-2995-4300-ac96-e7443b43403f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:40:59.467406 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63dfe5f-2995-4300-ac96-e7443b43403f service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:40:59.476529 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 738/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:40:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:41:01.737769 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a48e3-fad4-425d-8b71-4f6deed0bb7b None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:01.738693 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a48e3-fad4-425d-8b71-4f6deed0bb7b None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:01.738693 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a48e3-fad4-425d-8b71-4f6deed0bb7b None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:01.740858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 718/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 09:41:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:41:01.747334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cab1f1-6b59-4844-a6d6-6edcc7d8cefe None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:01.747756 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cab1f1-6b59-4844-a6d6-6edcc7d8cefe None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:01.747868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cab1f1-6b59-4844-a6d6-6edcc7d8cefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:01.767470 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-08cab1f1-6b59-4844-a6d6-6edcc7d8cefe None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:41:01.768651 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-08cab1f1-6b59-4844-a6d6-6edcc7d8cefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:41:01.777160 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 739/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 09:41:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:41:01.817500 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a48e3-fad4-425d-8b71-4f6deed0bb7b None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:41:01.831366 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a48e3-fad4-425d-8b71-4f6deed0bb7b None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:41:01.835765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a514cf-795c-406e-85e9-4a6191ec4219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:41:01.836462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a514cf-795c-406e-85e9-4a6191ec4219 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:01.836764 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a514cf-795c-406e-85e9-4a6191ec4219 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:01.837035 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a514cf-795c-406e-85e9-4a6191ec4219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:01.846148 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a514cf-795c-406e-85e9-4a6191ec4219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:41:01.847579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a514cf-795c-406e-85e9-4a6191ec4219 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:41:01.857738 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 719/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:41:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:41:04.116115 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a87f3-b87d-4968-a542-f0a9c114c1f9 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:04.116745 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a87f3-b87d-4968-a542-f0a9c114c1f9 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:04.116745 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a87f3-b87d-4968-a542-f0a9c114c1f9 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:04.118856 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 740/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 09:41:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:41:04.124520 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eac147-d0b8-48a3-b84b-8423ff8e96da None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:04.125028 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eac147-d0b8-48a3-b84b-8423ff8e96da None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:04.125147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eac147-d0b8-48a3-b84b-8423ff8e96da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:04.145005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-e3eac147-d0b8-48a3-b84b-8423ff8e96da None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:41:04.146201 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e3eac147-d0b8-48a3-b84b-8423ff8e96da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:41:04.154718 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 720/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 09:41:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:41:04.191398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a87f3-b87d-4968-a542-f0a9c114c1f9 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:41:04.202884 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a87f3-b87d-4968-a542-f0a9c114c1f9 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:41:04.207392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6821f-f59f-4713-adf3-1c4b30c1427a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:41:04.208035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6821f-f59f-4713-adf3-1c4b30c1427a service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:04.208311 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6821f-f59f-4713-adf3-1c4b30c1427a service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:04.208578 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6821f-f59f-4713-adf3-1c4b30c1427a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:04.218341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d6821f-f59f-4713-adf3-1c4b30c1427a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:41:04.219782 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d6821f-f59f-4713-adf3-1c4b30c1427a service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:41:04.229580 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 741/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:41:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:41:06.474514 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfd4db-5486-4271-bb11-68f11187dd4e None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:06.475100 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfd4db-5486-4271-bb11-68f11187dd4e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:06.475100 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfd4db-5486-4271-bb11-68f11187dd4e None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:06.476331 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 721/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 09:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:41:06.482695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a781f0-57d5-4100-92c6-4f5598db10f3 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:06.483162 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a781f0-57d5-4100-92c6-4f5598db10f3 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:06.483267 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a781f0-57d5-4100-92c6-4f5598db10f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:06.510051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-22a781f0-57d5-4100-92c6-4f5598db10f3 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:41:06.511287 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-22a781f0-57d5-4100-92c6-4f5598db10f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:41:06.520459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 742/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 09:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:41:06.558034 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfd4db-5486-4271-bb11-68f11187dd4e None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:41:06.572945 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfd4db-5486-4271-bb11-68f11187dd4e None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:41:06.576653 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8811d733-0ad8-4064-8bc9-cbfa552086c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:41:06.577316 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811d733-0ad8-4064-8bc9-cbfa552086c8 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:06.578547 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811d733-0ad8-4064-8bc9-cbfa552086c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:06.578547 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811d733-0ad8-4064-8bc9-cbfa552086c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:06.589936 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8811d733-0ad8-4064-8bc9-cbfa552086c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:41:06.591380 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8811d733-0ad8-4064-8bc9-cbfa552086c8 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:41:06.601033 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 722/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:41:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:41:08.818021 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64375c-a6d1-47cc-9c7a-8411a77ecdc0 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:08.818556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64375c-a6d1-47cc-9c7a-8411a77ecdc0 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:08.818556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64375c-a6d1-47cc-9c7a-8411a77ecdc0 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:08.820253 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 743/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 09:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:41:08.825816 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af543b17-0d67-4540-a5aa-39512afdb3d6 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:08.826288 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af543b17-0d67-4540-a5aa-39512afdb3d6 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:08.826397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af543b17-0d67-4540-a5aa-39512afdb3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:08.845031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-af543b17-0d67-4540-a5aa-39512afdb3d6 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:41:08.846251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-af543b17-0d67-4540-a5aa-39512afdb3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:41:08.854533 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 723/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 09:41:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:41:08.893928 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64375c-a6d1-47cc-9c7a-8411a77ecdc0 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:41:08.912777 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64375c-a6d1-47cc-9c7a-8411a77ecdc0 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:41:08.918025 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcef86-05eb-40d5-aa6e-2d6aca8f67c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:41:08.918661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcef86-05eb-40d5-aa6e-2d6aca8f67c7 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:08.918935 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcef86-05eb-40d5-aa6e-2d6aca8f67c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:08.919219 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcef86-05eb-40d5-aa6e-2d6aca8f67c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:08.928925 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbcef86-05eb-40d5-aa6e-2d6aca8f67c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:41:08.930497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbcef86-05eb-40d5-aa6e-2d6aca8f67c7 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:41:08.939907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 744/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:41:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:41:11.127983 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b7d8e-7c87-4a95-b3ac-c329aca8bb8c None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:11.128924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b7d8e-7c87-4a95-b3ac-c329aca8bb8c None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:11.128924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b7d8e-7c87-4a95-b3ac-c329aca8bb8c None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:11.131235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 724/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 09:41:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:41:11.138624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b174a-41be-4437-bb95-2925efd69719 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:11.139079 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b174a-41be-4437-bb95-2925efd69719 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:11.139172 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b174a-41be-4437-bb95-2925efd69719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:11.164950 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-b68b174a-41be-4437-bb95-2925efd69719 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:41:11.166163 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b68b174a-41be-4437-bb95-2925efd69719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:41:11.176916 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 745/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 09:41:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:41:11.213819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b7d8e-7c87-4a95-b3ac-c329aca8bb8c None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:41:11.227149 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b7d8e-7c87-4a95-b3ac-c329aca8bb8c None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:41:11.231147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39094a-0501-4961-be0c-0c9a93f86f41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:41:11.231784 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39094a-0501-4961-be0c-0c9a93f86f41 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:11.232060 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39094a-0501-4961-be0c-0c9a93f86f41 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:11.232325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39094a-0501-4961-be0c-0c9a93f86f41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:11.242104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f39094a-0501-4961-be0c-0c9a93f86f41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:41:11.243547 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f39094a-0501-4961-be0c-0c9a93f86f41 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:41:11.253065 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 725/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:41:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:41:44.032584 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc384200-41cf-41ae-b198-ae49d5287f7f None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:44.033280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc384200-41cf-41ae-b198-ae49d5287f7f None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:44.033280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc384200-41cf-41ae-b198-ae49d5287f7f None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:44.034512 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 746/1471] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:41:44.040500 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3d21e-38bf-4859-aaed-2567cc009ef1 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:44.041017 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3d21e-38bf-4859-aaed-2567cc009ef1 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:44.041017 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3d21e-38bf-4859-aaed-2567cc009ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:44.060565 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-8ef3d21e-38bf-4859-aaed-2567cc009ef1 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:41:44.061768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-8ef3d21e-38bf-4859-aaed-2567cc009ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:41:44.070993 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 726/1472] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:41:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:41:44.120429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b2c8e-4984-4289-9c61-4a67e2fc462d None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:44.120819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b2c8e-4984-4289-9c61-4a67e2fc462d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:44.121153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b2c8e-4984-4289-9c61-4a67e2fc462d None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:44.123202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 747/1473] 10.222.0.26 () {54 vars in 962 bytes} [Mon May 9 09:41:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:41:44.130558 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2451d77-86cf-456a-9b1e-df485a832787 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:44.130921 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2451d77-86cf-456a-9b1e-df485a832787 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:44.131252 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2451d77-86cf-456a-9b1e-df485a832787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:44.150617 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-a2451d77-86cf-456a-9b1e-df485a832787 None None] MFA Rules not processed for user `7fd398c547554629927d2a80b3a8dee6`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:41:44.151785 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a2451d77-86cf-456a-9b1e-df485a832787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:41:44.158554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 727/1474] 10.222.0.26 () {58 vars in 1059 bytes} [Mon May 9 09:41:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:41:44.166444 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b2c8e-4984-4289-9c61-4a67e2fc462d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:41:44.183280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b2c8e-4984-4289-9c61-4a67e2fc462d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:41:44.186921 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a52f1d-ac81-4e23-b6f1-86617a8fcf4f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fd398c547554629927d2a80b3a8dee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:41:44.187539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a52f1d-ac81-4e23-b6f1-86617a8fcf4f service glance] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:44.187811 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a52f1d-ac81-4e23-b6f1-86617a8fcf4f service glance] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:44.188078 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a52f1d-ac81-4e23-b6f1-86617a8fcf4f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:44.196884 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a52f1d-ac81-4e23-b6f1-86617a8fcf4f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:41:44.198294 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a52f1d-ac81-4e23-b6f1-86617a8fcf4f service glance] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:41:44.210163 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 748/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:41:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:41:46.515417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bb6d0-5a26-43ff-8efa-72c741fa2752 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:46.516285 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bb6d0-5a26-43ff-8efa-72c741fa2752 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:46.516285 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bb6d0-5a26-43ff-8efa-72c741fa2752 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:46.519186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 728/1476] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:41:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:41:46.528039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e601e92-c307-4dd2-afa5-d3a5bb4ffeca None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:46.528525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e601e92-c307-4dd2-afa5-d3a5bb4ffeca None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:46.528631 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e601e92-c307-4dd2-afa5-d3a5bb4ffeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:46.551025 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-9e601e92-c307-4dd2-afa5-d3a5bb4ffeca None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:41:46.552219 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-9e601e92-c307-4dd2-afa5-d3a5bb4ffeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:41:46.561545 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 749/1477] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:41:46.584915 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45bb6d0-5a26-43ff-8efa-72c741fa2752 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:41:46.598020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45bb6d0-5a26-43ff-8efa-72c741fa2752 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:41:46.602688 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10303555-8a10-424b-9184-448d4c6e9859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:41:46.603328 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10303555-8a10-424b-9184-448d4c6e9859 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:46.603604 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10303555-8a10-424b-9184-448d4c6e9859 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:46.603840 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10303555-8a10-424b-9184-448d4c6e9859 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:46.604574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10303555-8a10-424b-9184-448d4c6e9859 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:41:46.606375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: WARNING py.warnings [None req-10303555-8a10-424b-9184-448d4c6e9859 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 09:41:46.606375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: warnings.warn(msg) May 09 09:41:46.606375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: May 09 09:41:46.607511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10303555-8a10-424b-9184-448d4c6e9859 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:41:46.627128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 729/1478] 10.222.0.26 () {56 vars in 1108 bytes} [Mon May 9 09:41:46 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:41:48.844786 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1750eb-1230-4bb4-9e17-f5bf47ef2e0d None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:48.845361 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1750eb-1230-4bb4-9e17-f5bf47ef2e0d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:48.845361 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1750eb-1230-4bb4-9e17-f5bf47ef2e0d None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:48.846645 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 750/1479] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:41:48.852132 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825001aa-b744-4e31-a7fd-fb45e90e6f9c None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:48.852566 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825001aa-b744-4e31-a7fd-fb45e90e6f9c None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:48.852742 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825001aa-b744-4e31-a7fd-fb45e90e6f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:48.872246 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-825001aa-b744-4e31-a7fd-fb45e90e6f9c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:41:48.873479 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-825001aa-b744-4e31-a7fd-fb45e90e6f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:41:48.881734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 730/1480] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:41:48.923574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1750eb-1230-4bb4-9e17-f5bf47ef2e0d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:41:48.941622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1750eb-1230-4bb4-9e17-f5bf47ef2e0d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:41:48.948912 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939f9f2-3aed-4bdf-bab3-939bdb2dc96a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:41:48.949550 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939f9f2-3aed-4bdf-bab3-939bdb2dc96a service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:48.949853 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939f9f2-3aed-4bdf-bab3-939bdb2dc96a service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:48.950117 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939f9f2-3aed-4bdf-bab3-939bdb2dc96a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:48.958310 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939f9f2-3aed-4bdf-bab3-939bdb2dc96a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:41:48.959891 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939f9f2-3aed-4bdf-bab3-939bdb2dc96a service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:41:48.969486 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 751/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:41:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:41:51.478948 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b8daf-99d7-4700-bd6e-c7ca6a5fb912 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:51.479909 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b8daf-99d7-4700-bd6e-c7ca6a5fb912 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:51.479909 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b8daf-99d7-4700-bd6e-c7ca6a5fb912 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:51.482016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 731/1482] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:41:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:41:51.487857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0a89c-4a29-41ef-aff5-a40f9400ec65 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:51.488624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0a89c-4a29-41ef-aff5-a40f9400ec65 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:51.488778 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0a89c-4a29-41ef-aff5-a40f9400ec65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:51.508688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-dfe0a89c-4a29-41ef-aff5-a40f9400ec65 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:41:51.509898 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-dfe0a89c-4a29-41ef-aff5-a40f9400ec65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:41:51.518636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 752/1483] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:41:51.555640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b8daf-99d7-4700-bd6e-c7ca6a5fb912 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:41:51.568645 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b8daf-99d7-4700-bd6e-c7ca6a5fb912 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:41:51.573266 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2057b-2add-45d2-ba21-31337ade3bf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:41:51.574017 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2057b-2add-45d2-ba21-31337ade3bf7 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:51.574300 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2057b-2add-45d2-ba21-31337ade3bf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:51.574571 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2057b-2add-45d2-ba21-31337ade3bf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:51.583929 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2057b-2add-45d2-ba21-31337ade3bf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:41:51.585407 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2057b-2add-45d2-ba21-31337ade3bf7 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:41:51.595564 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 732/1484] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:41:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:41:53.832308 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae4668-3767-4a8c-b9b4-3a215ecec51b None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:53.832934 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae4668-3767-4a8c-b9b4-3a215ecec51b None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:53.832934 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae4668-3767-4a8c-b9b4-3a215ecec51b None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:53.834251 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 753/1485] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:41:53.839762 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aeee3c-30a4-4946-aab1-610ae4138c18 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:53.840267 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aeee3c-30a4-4946-aab1-610ae4138c18 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:53.840267 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aeee3c-30a4-4946-aab1-610ae4138c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:53.859540 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-e2aeee3c-30a4-4946-aab1-610ae4138c18 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:41:53.860711 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e2aeee3c-30a4-4946-aab1-610ae4138c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:41:53.869734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 733/1486] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:41:53.909378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae4668-3767-4a8c-b9b4-3a215ecec51b None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:41:53.923162 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae4668-3767-4a8c-b9b4-3a215ecec51b None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:41:53.927551 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e90f2-7a5b-4b25-a21b-a894dfb62651 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:41:53.928173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e90f2-7a5b-4b25-a21b-a894dfb62651 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:53.928448 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e90f2-7a5b-4b25-a21b-a894dfb62651 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:53.928743 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e90f2-7a5b-4b25-a21b-a894dfb62651 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:53.937862 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e90f2-7a5b-4b25-a21b-a894dfb62651 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:41:53.939293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e90f2-7a5b-4b25-a21b-a894dfb62651 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:41:53.948684 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 754/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:41:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:41:56.300745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a4721-c5ad-4f3f-be73-26959ebe838c None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:56.301555 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a4721-c5ad-4f3f-be73-26959ebe838c None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:56.301555 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a4721-c5ad-4f3f-be73-26959ebe838c None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:56.303202 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 734/1488] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:41:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:41:56.308537 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af7446-dd6f-4d1e-aa69-237c11fb050c None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:56.308999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af7446-dd6f-4d1e-aa69-237c11fb050c None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:56.309111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af7446-dd6f-4d1e-aa69-237c11fb050c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:56.331076 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-26af7446-dd6f-4d1e-aa69-237c11fb050c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:41:56.332275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-26af7446-dd6f-4d1e-aa69-237c11fb050c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:41:56.341638 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 755/1489] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:41:56.391296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a4721-c5ad-4f3f-be73-26959ebe838c None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:41:56.403275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a4721-c5ad-4f3f-be73-26959ebe838c None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:41:56.407934 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eedfb9-c48c-4f50-9e54-239b14877be8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:41:56.408567 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eedfb9-c48c-4f50-9e54-239b14877be8 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:56.408873 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eedfb9-c48c-4f50-9e54-239b14877be8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:56.409105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eedfb9-c48c-4f50-9e54-239b14877be8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:56.418198 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eedfb9-c48c-4f50-9e54-239b14877be8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:41:56.419631 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eedfb9-c48c-4f50-9e54-239b14877be8 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:41:56.428936 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 735/1490] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:41:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:41:58.851075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a022105-32e5-4dfd-93bb-37cce79f07fb None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:58.851960 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a022105-32e5-4dfd-93bb-37cce79f07fb None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:58.851960 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a022105-32e5-4dfd-93bb-37cce79f07fb None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:58.854097 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 756/1491] 10.222.0.26 () {54 vars in 926 bytes} [Mon May 9 09:41:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:41:58.859663 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951da9fb-2877-4d71-bde3-2faf5266fb44 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:58.860143 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951da9fb-2877-4d71-bde3-2faf5266fb44 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:58.860272 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951da9fb-2877-4d71-bde3-2faf5266fb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:58.879045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-951da9fb-2877-4d71-bde3-2faf5266fb44 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:41:58.880235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-951da9fb-2877-4d71-bde3-2faf5266fb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:41:58.888535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 736/1492] 10.222.0.26 () {58 vars in 1023 bytes} [Mon May 9 09:41:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:41:58.929752 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a022105-32e5-4dfd-93bb-37cce79f07fb None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:41:58.942031 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a022105-32e5-4dfd-93bb-37cce79f07fb None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:41:58.946347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748841f3-58e2-4769-8be0-b677d10e6805 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:41:58.946984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748841f3-58e2-4769-8be0-b677d10e6805 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:41:58.947265 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748841f3-58e2-4769-8be0-b677d10e6805 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:41:58.947538 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748841f3-58e2-4769-8be0-b677d10e6805 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:41:58.956864 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748841f3-58e2-4769-8be0-b677d10e6805 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:41:58.958304 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748841f3-58e2-4769-8be0-b677d10e6805 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:41:58.967656 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 757/1493] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:41:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:43:29.477626 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c1785-77f6-467c-90cc-37089c9b5ae1 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:43:29.478182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c1785-77f6-467c-90cc-37089c9b5ae1 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:43:29.478312 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c1785-77f6-467c-90cc-37089c9b5ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:43:29.499698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-9c1c1785-77f6-467c-90cc-37089c9b5ae1 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:43:29.500938 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-9c1c1785-77f6-467c-90cc-37089c9b5ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:43:29.509811 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 737/1494] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:43:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:43:29.519508 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78949164-cf1d-4584-8c14-ec07448c6f74 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:43:29.520051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78949164-cf1d-4584-8c14-ec07448c6f74 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:43:29.520051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78949164-cf1d-4584-8c14-ec07448c6f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:43:29.538216 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-78949164-cf1d-4584-8c14-ec07448c6f74 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:43:29.539422 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-78949164-cf1d-4584-8c14-ec07448c6f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:43:29.548490 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 758/1495] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:43:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:43:29.553242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c1785-77f6-467c-90cc-37089c9b5ae1 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:43:29.564780 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c1785-77f6-467c-90cc-37089c9b5ae1 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:43:29.570099 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c2f04-4441-437f-bd83-1231ebc2ce1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:43:29.570776 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c2f04-4441-437f-bd83-1231ebc2ce1a admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:43:29.571038 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c2f04-4441-437f-bd83-1231ebc2ce1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:43:29.571305 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c2f04-4441-437f-bd83-1231ebc2ce1a admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:43:29.572033 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c2f04-4441-437f-bd83-1231ebc2ce1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:43:29.572332 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c2f04-4441-437f-bd83-1231ebc2ce1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:43:29.573390 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: WARNING py.warnings [None req-117c2f04-4441-437f-bd83-1231ebc2ce1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 09 09:43:29.573390 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: warnings.warn(msg) May 09 09:43:29.573390 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: May 09 09:43:29.574337 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c2f04-4441-437f-bd83-1231ebc2ce1a admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:43:29.586233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 738/1496] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:43:29.594776 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78949164-cf1d-4584-8c14-ec07448c6f74 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:43:29.605068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78949164-cf1d-4584-8c14-ec07448c6f74 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:43:29.609585 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80002f-289d-4bf8-8ae2-7729520c0074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:43:29.610231 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80002f-289d-4bf8-8ae2-7729520c0074 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:43:29.610504 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80002f-289d-4bf8-8ae2-7729520c0074 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:43:29.610765 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80002f-289d-4bf8-8ae2-7729520c0074 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:43:29.611470 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b80002f-289d-4bf8-8ae2-7729520c0074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:43:29.613291 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: WARNING py.warnings [None req-9b80002f-289d-4bf8-8ae2-7729520c0074 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 09:43:29.613291 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: warnings.warn(msg) May 09 09:43:29.613291 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: May 09 09:43:29.614812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b80002f-289d-4bf8-8ae2-7729520c0074 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:43:29.650750 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 759/1497] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:43:29 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:43:29.655269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c2f04-4441-437f-bd83-1231ebc2ce1a admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:43:29.665194 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c2f04-4441-437f-bd83-1231ebc2ce1a admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:43:29.722028 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e9af3-c6df-440c-ab3b-465532847257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:43:29.722818 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e9af3-c6df-440c-ab3b-465532847257 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:43:29.723149 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e9af3-c6df-440c-ab3b-465532847257 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:43:29.723473 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e9af3-c6df-440c-ab3b-465532847257 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:43:29.724526 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e9af3-c6df-440c-ab3b-465532847257 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:43:29.726123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: WARNING py.warnings [None req-1b7e9af3-c6df-440c-ab3b-465532847257 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 09:43:29.726123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: warnings.warn(msg) May 09 09:43:29.726123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: May 09 09:43:29.727682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e9af3-c6df-440c-ab3b-465532847257 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:43:29.757137 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 739/1498] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:43:29 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:43:29.761300 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80002f-289d-4bf8-8ae2-7729520c0074 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:43:29.771522 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80002f-289d-4bf8-8ae2-7729520c0074 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:43:29.777243 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d496a9-4bde-4e34-b973-c404bd1d330c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:43:29.777897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d496a9-4bde-4e34-b973-c404bd1d330c admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:43:29.778159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d496a9-4bde-4e34-b973-c404bd1d330c admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:43:29.778446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d496a9-4bde-4e34-b973-c404bd1d330c admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:43:29.779165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d496a9-4bde-4e34-b973-c404bd1d330c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:43:29.780185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: WARNING py.warnings [None req-02d496a9-4bde-4e34-b973-c404bd1d330c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 09 09:43:29.780185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: warnings.warn(msg) May 09 09:43:29.780185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: May 09 09:43:29.781138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d496a9-4bde-4e34-b973-c404bd1d330c admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:43:29.794675 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 760/1499] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:43:29.798929 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e9af3-c6df-440c-ab3b-465532847257 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:43:29.809495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e9af3-c6df-440c-ab3b-465532847257 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:43:29.815733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13285231-3e47-4ac7-83ca-5c55064b6970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:43:29.816383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13285231-3e47-4ac7-83ca-5c55064b6970 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:43:29.816678 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13285231-3e47-4ac7-83ca-5c55064b6970 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:43:29.816963 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13285231-3e47-4ac7-83ca-5c55064b6970 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:43:29.817668 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13285231-3e47-4ac7-83ca-5c55064b6970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:43:29.818688 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: WARNING py.warnings [None req-13285231-3e47-4ac7-83ca-5c55064b6970 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 09 09:43:29.818688 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: warnings.warn(msg) May 09 09:43:29.818688 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: May 09 09:43:29.819675 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13285231-3e47-4ac7-83ca-5c55064b6970 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:43:29.833247 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 740/1500] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:43:29.837463 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d496a9-4bde-4e34-b973-c404bd1d330c admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:43:29.847727 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d496a9-4bde-4e34-b973-c404bd1d330c admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:43:29.853559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731adde-41f1-4a90-b810-37e3101c83ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:43:29.854432 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731adde-41f1-4a90-b810-37e3101c83ac admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:43:29.854705 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731adde-41f1-4a90-b810-37e3101c83ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:43:29.854940 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731adde-41f1-4a90-b810-37e3101c83ac admin admin] PATH_INFO: `/v3/projects/6c633ccf0cf847d9ad4f81198eab1aa6/users/47046921e6074edeb24e81234d02be82/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:43:29.870615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731adde-41f1-4a90-b810-37e3101c83ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c633ccf0cf847d9ad4f81198eab1aa6, user_id=47046921e6074edeb24e81234d02be82, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:43:29.871647 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: WARNING py.warnings [None req-8731adde-41f1-4a90-b810-37e3101c83ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 09:43:29.871647 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: warnings.warn(msg) May 09 09:43:29.871647 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: May 09 09:43:29.873605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731adde-41f1-4a90-b810-37e3101c83ac admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:43:29.888148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 761/1501] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:43:29 2022] PUT /identity/v3/projects/6c633ccf0cf847d9ad4f81198eab1aa6/users/47046921e6074edeb24e81234d02be82/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:43:29.899373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ee0c4-7345-4ece-91b1-3a674bbd808e None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:43:29.899668 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ee0c4-7345-4ece-91b1-3a674bbd808e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:43:29.899932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ee0c4-7345-4ece-91b1-3a674bbd808e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:43:29.927127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-cc5ee0c4-7345-4ece-91b1-3a674bbd808e None None] MFA Rules not processed for user `47046921e6074edeb24e81234d02be82`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:43:29.928333 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-cc5ee0c4-7345-4ece-91b1-3a674bbd808e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:43:29.997052 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 741/1502] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:43:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:43:30.011317 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d48b7-d59e-4462-b215-ccf8ddac27fe None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:43:30.011671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d48b7-d59e-4462-b215-ccf8ddac27fe None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:43:30.012248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d48b7-d59e-4462-b215-ccf8ddac27fe None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:43:30.014362 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 762/1503] 10.222.0.26 () {54 vars in 963 bytes} [Mon May 9 09:43:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:43:30.019376 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bf7d8-fdfe-40cf-ade5-6442b76f0920 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:43:30.019660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bf7d8-fdfe-40cf-ade5-6442b76f0920 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:43:30.019931 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bf7d8-fdfe-40cf-ade5-6442b76f0920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:43:30.042027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-a76bf7d8-fdfe-40cf-ade5-6442b76f0920 None None] MFA Rules not processed for user `a44d689993bd4d30858a770012ee3e89`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:43:30.043197 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a76bf7d8-fdfe-40cf-ade5-6442b76f0920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:43:30.095095 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 742/1504] 10.222.0.26 () {58 vars in 1060 bytes} [Mon May 9 09:43:30 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:43:30.102578 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d48b7-d59e-4462-b215-ccf8ddac27fe None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:43:30.114466 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d48b7-d59e-4462-b215-ccf8ddac27fe None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:43:30.118563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec47f0f-813c-47ed-ae83-ff71345fe7f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:43:30.119195 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec47f0f-813c-47ed-ae83-ff71345fe7f6 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:43:30.119468 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec47f0f-813c-47ed-ae83-ff71345fe7f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:43:30.119733 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec47f0f-813c-47ed-ae83-ff71345fe7f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:43:30.128903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec47f0f-813c-47ed-ae83-ff71345fe7f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:43:30.130317 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec47f0f-813c-47ed-ae83-ff71345fe7f6 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:43:30.139380 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 763/1505] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:43:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:43:30.181969 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e14d81-fc8c-47e7-8f4d-52a58c4a8d5d None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:43:30.182255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e14d81-fc8c-47e7-8f4d-52a58c4a8d5d None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:43:30.182522 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e14d81-fc8c-47e7-8f4d-52a58c4a8d5d None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:43:30.183787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 743/1506] 10.222.0.26 () {52 vars in 829 bytes} [Mon May 9 09:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:43:30.187501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec47f0f-813c-47ed-ae83-ff71345fe7f6 service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:43:30.251406 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec47f0f-813c-47ed-ae83-ff71345fe7f6 service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:43:30.257127 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb3b42-8b05-493f-b75c-590c39db2d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:43:30.257998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb3b42-8b05-493f-b75c-590c39db2d5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:43:30.258278 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb3b42-8b05-493f-b75c-590c39db2d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:43:30.258549 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb3b42-8b05-493f-b75c-590c39db2d5e admin admin] PATH_INFO: `/v3/users/47046921e6074edeb24e81234d02be82` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:43:30.259860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedb3b42-8b05-493f-b75c-590c39db2d5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=47046921e6074edeb24e81234d02be82)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:43:30.260935 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: WARNING py.warnings [None req-dedb3b42-8b05-493f-b75c-590c39db2d5e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 09:43:30.260935 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: warnings.warn(msg) May 09 09:43:30.260935 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: May 09 09:43:30.262196 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedb3b42-8b05-493f-b75c-590c39db2d5e admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:43:30.329034 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-dedb3b42-8b05-493f-b75c-590c39db2d5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47046921e6074edeb24e81234d02be82'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:43:30.342803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-dedb3b42-8b05-493f-b75c-590c39db2d5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47046921e6074edeb24e81234d02be82'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:43:30.343570 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-dedb3b42-8b05-493f-b75c-590c39db2d5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47046921e6074edeb24e81234d02be82'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:43:30.354365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-dedb3b42-8b05-493f-b75c-590c39db2d5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47046921e6074edeb24e81234d02be82'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:43:30.355040 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-dedb3b42-8b05-493f-b75c-590c39db2d5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47046921e6074edeb24e81234d02be82'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:43:30.505572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 764/1507] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:43:30 2022] DELETE /identity/v3/users/47046921e6074edeb24e81234d02be82 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:43:30.510131 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0b385-f1ff-4325-b084-6f6323837643 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:43:30.510629 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0b385-f1ff-4325-b084-6f6323837643 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:43:30.510629 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0b385-f1ff-4325-b084-6f6323837643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:43:30.529328 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-26c0b385-f1ff-4325-b084-6f6323837643 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:43:30.530505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-26c0b385-f1ff-4325-b084-6f6323837643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:43:30.593449 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 744/1508] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:43:30.604901 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb3b42-8b05-493f-b75c-590c39db2d5e admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:43:30.606604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-dedb3b42-8b05-493f-b75c-590c39db2d5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:43:30.683010 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb3b42-8b05-493f-b75c-590c39db2d5e admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:43:30.687402 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45135556-7f84-437e-8d8d-524a5189b732 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:43:30.688036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45135556-7f84-437e-8d8d-524a5189b732 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:43:30.688315 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45135556-7f84-437e-8d8d-524a5189b732 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:43:30.688561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45135556-7f84-437e-8d8d-524a5189b732 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:43:30.698048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45135556-7f84-437e-8d8d-524a5189b732 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:43:30.699488 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45135556-7f84-437e-8d8d-524a5189b732 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:43:30.708971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 765/1509] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:43:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:43:31.067675 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0b385-f1ff-4325-b084-6f6323837643 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:43:31.069255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-26c0b385-f1ff-4325-b084-6f6323837643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:43:31.081136 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0b385-f1ff-4325-b084-6f6323837643 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:43:31.086826 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0a42e-8f0c-4f65-80f3-4dfe88aafd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:43:31.087819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0a42e-8f0c-4f65-80f3-4dfe88aafd84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:43:31.088194 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0a42e-8f0c-4f65-80f3-4dfe88aafd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:43:31.088556 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0a42e-8f0c-4f65-80f3-4dfe88aafd84 admin admin] PATH_INFO: `/v3/projects/6c633ccf0cf847d9ad4f81198eab1aa6` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:43:31.089969 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0a42e-8f0c-4f65-80f3-4dfe88aafd84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c633ccf0cf847d9ad4f81198eab1aa6)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:43:31.091141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: WARNING py.warnings [None req-00c0a42e-8f0c-4f65-80f3-4dfe88aafd84 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 09:43:31.091141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: warnings.warn(msg) May 09 09:43:31.091141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: May 09 09:43:31.092505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0a42e-8f0c-4f65-80f3-4dfe88aafd84 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:43:31.154059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-00c0a42e-8f0c-4f65-80f3-4dfe88aafd84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c633ccf0cf847d9ad4f81198eab1aa6', 'request_id': 'req-00c0a42e-8f0c-4f65-80f3-4dfe88aafd84'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:43:31.167651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-00c0a42e-8f0c-4f65-80f3-4dfe88aafd84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c633ccf0cf847d9ad4f81198eab1aa6', 'request_id': 'req-00c0a42e-8f0c-4f65-80f3-4dfe88aafd84'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:43:31.174262 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-00c0a42e-8f0c-4f65-80f3-4dfe88aafd84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c633ccf0cf847d9ad4f81198eab1aa6', 'request_id': 'req-00c0a42e-8f0c-4f65-80f3-4dfe88aafd84'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:43:31.188977 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-00c0a42e-8f0c-4f65-80f3-4dfe88aafd84 admin admin] The token cache is being invalidate because project 6c633ccf0cf847d9ad4f81198eab1aa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:43:31.189269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-00c0a42e-8f0c-4f65-80f3-4dfe88aafd84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:43:31.189988 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-00c0a42e-8f0c-4f65-80f3-4dfe88aafd84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:43:31.191587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 745/1510] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:43:31 2022] DELETE /identity/v3/projects/6c633ccf0cf847d9ad4f81198eab1aa6 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:45:12.821896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d93f96-3c95-48bc-a1ab-790ba181d4b5 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:12.822495 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d93f96-3c95-48bc-a1ab-790ba181d4b5 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:12.822495 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d93f96-3c95-48bc-a1ab-790ba181d4b5 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:12.823766 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 766/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 09:45:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:45:12.829686 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6966d69-fc3b-4067-9d74-0679c75f3aa1 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:12.830496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6966d69-fc3b-4067-9d74-0679c75f3aa1 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:12.830496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6966d69-fc3b-4067-9d74-0679c75f3aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:12.855519 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-f6966d69-fc3b-4067-9d74-0679c75f3aa1 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:45:12.856752 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f6966d69-fc3b-4067-9d74-0679c75f3aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:45:12.921579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 746/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 09:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:45:13.098903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d93f96-3c95-48bc-a1ab-790ba181d4b5 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:45:13.101659 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-90d93f96-3c95-48bc-a1ab-790ba181d4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:45:13.185912 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d93f96-3c95-48bc-a1ab-790ba181d4b5 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:45:13.193437 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818a9ac-349f-4bc7-abeb-bd7a98ea2c82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:45:13.194088 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818a9ac-349f-4bc7-abeb-bd7a98ea2c82 service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:13.194363 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818a9ac-349f-4bc7-abeb-bd7a98ea2c82 service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:13.194627 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818a9ac-349f-4bc7-abeb-bd7a98ea2c82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:13.203564 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4818a9ac-349f-4bc7-abeb-bd7a98ea2c82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:45:13.205234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4818a9ac-349f-4bc7-abeb-bd7a98ea2c82 service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:45:13.215277 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 767/1513] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:45:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:45:14.109503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c38a6-a9bc-4168-9fe4-099c4b430cc2 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:14.110182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c38a6-a9bc-4168-9fe4-099c4b430cc2 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:14.110182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c38a6-a9bc-4168-9fe4-099c4b430cc2 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:14.111419 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 747/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 09:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:45:14.117375 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fd064-c715-4b9a-b149-2faa6abfb808 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:14.117973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fd064-c715-4b9a-b149-2faa6abfb808 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:14.118080 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fd064-c715-4b9a-b149-2faa6abfb808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:14.139963 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-315fd064-c715-4b9a-b149-2faa6abfb808 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:45:14.141219 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-315fd064-c715-4b9a-b149-2faa6abfb808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:45:14.150446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 768/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 09:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:45:14.221414 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7de360-b7ca-4e90-9ac5-f160e0c3fbd5 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:14.221682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7de360-b7ca-4e90-9ac5-f160e0c3fbd5 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:14.221975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7de360-b7ca-4e90-9ac5-f160e0c3fbd5 None None] PATH_INFO: `/v3` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:14.223375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 748/1516] 10.222.0.50 () {54 vars in 925 bytes} [Mon May 9 09:45:14 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:45:14.230938 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3652a-3ca9-476d-9afd-842d82c8dca1 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:14.231224 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3652a-3ca9-476d-9afd-842d82c8dca1 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:14.231494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3652a-3ca9-476d-9afd-842d82c8dca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:14.258181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-56c3652a-3ca9-476d-9afd-842d82c8dca1 None None] MFA Rules not processed for user `6c9e458c2c5f4ed1b3eddd84c2ec1703`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:45:14.259409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-56c3652a-3ca9-476d-9afd-842d82c8dca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:45:14.310831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7de360-b7ca-4e90-9ac5-f160e0c3fbd5 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:45:14.322998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 769/1517] 10.222.0.50 () {58 vars in 1015 bytes} [Mon May 9 09:45:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:45:14.325418 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7de360-b7ca-4e90-9ac5-f160e0c3fbd5 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:45:14.331080 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292ab6a-4e70-41e5-a2a4-b957514b3144 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:45:14.331716 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292ab6a-4e70-41e5-a2a4-b957514b3144 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:14.331996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292ab6a-4e70-41e5-a2a4-b957514b3144 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:14.332268 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292ab6a-4e70-41e5-a2a4-b957514b3144 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:14.340934 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5292ab6a-4e70-41e5-a2a4-b957514b3144 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:45:14.342558 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5292ab6a-4e70-41e5-a2a4-b957514b3144 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:45:14.352026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 749/1518] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:45:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:45:24.419292 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8ded1-d1c4-4cf4-8041-4f9eae63513d None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:24.419910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8ded1-d1c4-4cf4-8041-4f9eae63513d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:24.419910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8ded1-d1c4-4cf4-8041-4f9eae63513d None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:24.421222 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 770/1519] 10.222.0.26 () {54 vars in 960 bytes} [Mon May 9 09:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:45:24.426380 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a44d9-8752-44b4-91e6-01ff168f983a None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:24.426940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a44d9-8752-44b4-91e6-01ff168f983a None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:24.426940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a44d9-8752-44b4-91e6-01ff168f983a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:24.446733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-142a44d9-8752-44b4-91e6-01ff168f983a None None] MFA Rules not processed for user `6c9e458c2c5f4ed1b3eddd84c2ec1703`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:45:24.447945 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-142a44d9-8752-44b4-91e6-01ff168f983a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:45:24.457902 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 750/1520] 10.222.0.26 () {58 vars in 1057 bytes} [Mon May 9 09:45:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:45:24.466827 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8ded1-d1c4-4cf4-8041-4f9eae63513d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:45:24.480234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8ded1-d1c4-4cf4-8041-4f9eae63513d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:45:24.485194 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a5008-af24-4fcb-b7d3-f4f5955f232d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9e458c2c5f4ed1b3eddd84c2ec1703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:45:24.485881 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a5008-af24-4fcb-b7d3-f4f5955f232d service placement] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:24.486147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a5008-af24-4fcb-b7d3-f4f5955f232d service placement] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:24.486420 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a5008-af24-4fcb-b7d3-f4f5955f232d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:24.494980 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042a5008-af24-4fcb-b7d3-f4f5955f232d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:45:24.496555 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042a5008-af24-4fcb-b7d3-f4f5955f232d service placement] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:45:24.506015 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 771/1521] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:45:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:45:25.698708 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c172ab-8157-4ec3-ae5c-605ab955173b None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:25.699264 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c172ab-8157-4ec3-ae5c-605ab955173b None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:25.699264 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c172ab-8157-4ec3-ae5c-605ab955173b None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:25.700689 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 751/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 09:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:45:25.705755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03939be4-1295-41b0-953f-c1235c55ff57 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:25.706213 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03939be4-1295-41b0-953f-c1235c55ff57 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:25.706340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03939be4-1295-41b0-953f-c1235c55ff57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:25.724611 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-03939be4-1295-41b0-953f-c1235c55ff57 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:45:25.725854 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-03939be4-1295-41b0-953f-c1235c55ff57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:45:25.734212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 772/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 09:45:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:45:25.890271 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c172ab-8157-4ec3-ae5c-605ab955173b None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:45:25.915404 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c172ab-8157-4ec3-ae5c-605ab955173b None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:45:25.925512 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de232547-7e31-468e-9ae4-4c7b74b09d18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:45:25.926541 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de232547-7e31-468e-9ae4-4c7b74b09d18 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:25.926824 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de232547-7e31-468e-9ae4-4c7b74b09d18 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:25.927085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de232547-7e31-468e-9ae4-4c7b74b09d18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:25.939100 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de232547-7e31-468e-9ae4-4c7b74b09d18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:45:25.941588 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de232547-7e31-468e-9ae4-4c7b74b09d18 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:45:25.955860 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 752/1524] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:45:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:45:26.839961 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf42ca-362b-48a0-92e4-c2ebbeba61eb None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:26.840522 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf42ca-362b-48a0-92e4-c2ebbeba61eb None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:26.840522 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf42ca-362b-48a0-92e4-c2ebbeba61eb None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:26.842144 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 773/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 09:45:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:45:26.848415 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f17b0-f6ec-4a42-988a-b7fee132ed76 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:26.848883 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f17b0-f6ec-4a42-988a-b7fee132ed76 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:26.848992 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f17b0-f6ec-4a42-988a-b7fee132ed76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:26.869882 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-de2f17b0-f6ec-4a42-988a-b7fee132ed76 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:45:26.871094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-de2f17b0-f6ec-4a42-988a-b7fee132ed76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:45:26.880273 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 753/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 09:45:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:45:27.038938 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf42ca-362b-48a0-92e4-c2ebbeba61eb None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:45:27.057984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf42ca-362b-48a0-92e4-c2ebbeba61eb None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:45:27.067126 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d7731-6654-49a8-a65c-7ff06807a6f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:45:27.067766 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d7731-6654-49a8-a65c-7ff06807a6f2 service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:27.068034 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d7731-6654-49a8-a65c-7ff06807a6f2 service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:27.068272 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d7731-6654-49a8-a65c-7ff06807a6f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:27.077771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d7731-6654-49a8-a65c-7ff06807a6f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:45:27.079361 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d7731-6654-49a8-a65c-7ff06807a6f2 service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:45:27.089420 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 774/1527] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:45:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:45:31.251841 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffc5f2-67f0-4b4e-9134-5d508f3ee3d6 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:31.252472 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffc5f2-67f0-4b4e-9134-5d508f3ee3d6 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:31.252472 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffc5f2-67f0-4b4e-9134-5d508f3ee3d6 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:31.253820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 754/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 09:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:45:31.259298 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3d9b3-cc6f-46be-bdb7-d5ade94d909a None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:31.259770 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3d9b3-cc6f-46be-bdb7-d5ade94d909a None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:31.259865 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3d9b3-cc6f-46be-bdb7-d5ade94d909a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:31.287441 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-1fd3d9b3-cc6f-46be-bdb7-d5ade94d909a None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:45:31.288696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-1fd3d9b3-cc6f-46be-bdb7-d5ade94d909a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:45:31.298059 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 775/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 09:45:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:45:31.335619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ffc5f2-67f0-4b4e-9134-5d508f3ee3d6 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:45:31.337701 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-c5ffc5f2-67f0-4b4e-9134-5d508f3ee3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:45:31.401832 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ffc5f2-67f0-4b4e-9134-5d508f3ee3d6 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:45:31.406638 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c953d9-7628-4943-98e4-c451146a68da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:45:31.407283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c953d9-7628-4943-98e4-c451146a68da service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:31.407555 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c953d9-7628-4943-98e4-c451146a68da service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:31.407826 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c953d9-7628-4943-98e4-c451146a68da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:31.417037 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c953d9-7628-4943-98e4-c451146a68da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:45:31.418477 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c953d9-7628-4943-98e4-c451146a68da service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:45:31.427819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 755/1530] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:45:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 09:45:33.609459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3481d1-d1ee-4408-9899-77456e5f45af None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:33.610292 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3481d1-d1ee-4408-9899-77456e5f45af None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:33.610292 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3481d1-d1ee-4408-9899-77456e5f45af None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:33.612552 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 776/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 09:45:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:45:33.619236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79135c-c703-4619-89d4-53997402208c None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:33.620044 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79135c-c703-4619-89d4-53997402208c None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:33.620044 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79135c-c703-4619-89d4-53997402208c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:33.641513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-2b79135c-c703-4619-89d4-53997402208c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:45:33.642712 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-2b79135c-c703-4619-89d4-53997402208c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:45:33.651035 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 756/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 09:45:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:45:33.685946 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3481d1-d1ee-4408-9899-77456e5f45af None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:45:33.697487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3481d1-d1ee-4408-9899-77456e5f45af None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:45:33.701121 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7526dc-9969-40f7-bb97-578843107152 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:45:33.701762 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7526dc-9969-40f7-bb97-578843107152 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:33.702072 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7526dc-9969-40f7-bb97-578843107152 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:33.702344 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7526dc-9969-40f7-bb97-578843107152 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:33.720539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7526dc-9969-40f7-bb97-578843107152 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:45:33.722022 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7526dc-9969-40f7-bb97-578843107152 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:45:33.731304 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 777/1533] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:45:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:45:37.810657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d6d08-cb3b-40b4-b7da-668e36848d2e None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:37.811493 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d6d08-cb3b-40b4-b7da-668e36848d2e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:37.811493 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d6d08-cb3b-40b4-b7da-668e36848d2e None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:37.813882 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 757/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 09:45:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:45:37.820196 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d619c-7058-4a8b-83a8-b52ae32159cb None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:37.820776 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d619c-7058-4a8b-83a8-b52ae32159cb None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:37.820776 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d619c-7058-4a8b-83a8-b52ae32159cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:37.843052 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-a23d619c-7058-4a8b-83a8-b52ae32159cb None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:45:37.844261 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-a23d619c-7058-4a8b-83a8-b52ae32159cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:45:37.852785 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 778/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 09:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:45:38.128623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d6d08-cb3b-40b4-b7da-668e36848d2e None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:45:38.131001 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a41d6d08-cb3b-40b4-b7da-668e36848d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:45:38.193735 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d6d08-cb3b-40b4-b7da-668e36848d2e None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:45:38.198181 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad67e65-f08d-42aa-a6a9-c994eb55d8c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:45:38.198832 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad67e65-f08d-42aa-a6a9-c994eb55d8c0 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:38.199105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad67e65-f08d-42aa-a6a9-c994eb55d8c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:38.199372 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad67e65-f08d-42aa-a6a9-c994eb55d8c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:38.209586 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad67e65-f08d-42aa-a6a9-c994eb55d8c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:45:38.211035 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad67e65-f08d-42aa-a6a9-c994eb55d8c0 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:45:38.220825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 758/1536] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:45:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:45:40.478425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cd161-49de-4c92-bb61-051e591dc266 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:40.478985 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cd161-49de-4c92-bb61-051e591dc266 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:40.478985 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cd161-49de-4c92-bb61-051e591dc266 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:40.480265 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 779/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 09:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:45:40.486807 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f5389-7ff8-4b18-8ac2-1259764555fa None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:40.487561 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f5389-7ff8-4b18-8ac2-1259764555fa None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:40.487693 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f5389-7ff8-4b18-8ac2-1259764555fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:40.512628 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-9a5f5389-7ff8-4b18-8ac2-1259764555fa None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:45:40.513858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-9a5f5389-7ff8-4b18-8ac2-1259764555fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:45:40.522652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 759/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 09:45:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:45:40.792346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cd161-49de-4c92-bb61-051e591dc266 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:45:40.794383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e71cd161-49de-4c92-bb61-051e591dc266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:45:40.808434 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cd161-49de-4c92-bb61-051e591dc266 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:45:40.812301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d7905-2bff-44d5-bc2d-01d15ad64334 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:45:40.812965 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d7905-2bff-44d5-bc2d-01d15ad64334 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:45:40.813242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d7905-2bff-44d5-bc2d-01d15ad64334 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:45:40.813509 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d7905-2bff-44d5-bc2d-01d15ad64334 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:45:40.822624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d7905-2bff-44d5-bc2d-01d15ad64334 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:45:40.824062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d7905-2bff-44d5-bc2d-01d15ad64334 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:45:40.833434 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 780/1539] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:45:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:11.896193 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7f500-abe7-4464-aee4-8aa0c4a85fef None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:11.896885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7f500-abe7-4464-aee4-8aa0c4a85fef None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:11.896885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7f500-abe7-4464-aee4-8aa0c4a85fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:11.915575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113e613-27f8-4be6-81ea-6d7398397d05 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:11.916701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113e613-27f8-4be6-81ea-6d7398397d05 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:11.917197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113e613-27f8-4be6-81ea-6d7398397d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:11.919660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-7ae7f500-abe7-4464-aee4-8aa0c4a85fef None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:11.920854 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-7ae7f500-abe7-4464-aee4-8aa0c4a85fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:11.937157 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 760/1540] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:11.942819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-1113e613-27f8-4be6-81ea-6d7398397d05 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:11.944379 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-1113e613-27f8-4be6-81ea-6d7398397d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:11.951465 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da92b52-eb7a-4176-bfe2-188717883de8 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:11.951706 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da92b52-eb7a-4176-bfe2-188717883de8 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:11.951954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da92b52-eb7a-4176-bfe2-188717883de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:11.961078 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 781/1541] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:11.981019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f663b3-a1cb-4962-b9fa-8559743c45af None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:11.981679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f663b3-a1cb-4962-b9fa-8559743c45af None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:11.982169 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f663b3-a1cb-4962-b9fa-8559743c45af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:11.986212 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-5da92b52-eb7a-4176-bfe2-188717883de8 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:11.987758 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-5da92b52-eb7a-4176-bfe2-188717883de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:11.998112 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 761/1542] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:47:11.999480 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04102178-42c6-41a0-9c2b-373961170c7d None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:11.999759 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04102178-42c6-41a0-9c2b-373961170c7d None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.000026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04102178-42c6-41a0-9c2b-373961170c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.006466 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-75f663b3-a1cb-4962-b9fa-8559743c45af None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:12.008208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-75f663b3-a1cb-4962-b9fa-8559743c45af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:12.022351 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 782/1543] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:47:12.024059 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0a6d3-4248-434b-8a88-2753bdd98dcf None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.024576 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0a6d3-4248-434b-8a88-2753bdd98dcf None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.025014 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-04102178-42c6-41a0-9c2b-373961170c7d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:12.025337 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0a6d3-4248-434b-8a88-2753bdd98dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.028475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-04102178-42c6-41a0-9c2b-373961170c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:12.047578 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 762/1544] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:12.048742 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04102178-42c6-41a0-9c2b-373961170c7d None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.049271 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-7cb0a6d3-4248-434b-8a88-2753bdd98dcf None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:12.050862 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-7cb0a6d3-4248-434b-8a88-2753bdd98dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:12.061055 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04102178-42c6-41a0-9c2b-373961170c7d None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:12.070045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 783/1545] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:12.070871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463fdf5-5f6f-4397-9dcd-0a356dd93c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:12.071738 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463fdf5-5f6f-4397-9dcd-0a356dd93c46 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.072846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0a6d3-4248-434b-8a88-2753bdd98dcf None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.073332 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463fdf5-5f6f-4397-9dcd-0a356dd93c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.073758 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463fdf5-5f6f-4397-9dcd-0a356dd93c46 admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.074709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463fdf5-5f6f-4397-9dcd-0a356dd93c46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:47:12.077134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463fdf5-5f6f-4397-9dcd-0a356dd93c46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:12.079224 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463fdf5-5f6f-4397-9dcd-0a356dd93c46 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:12.094902 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0a6d3-4248-434b-8a88-2753bdd98dcf None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:12.101585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 763/1546] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:47:12.103177 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db9f21-a90c-43b8-b083-addeb6d424f5 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.103622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db9f21-a90c-43b8-b083-addeb6d424f5 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.104052 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db9f21-a90c-43b8-b083-addeb6d424f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.106109 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed73efd-16ad-4ae4-b3d4-8f0624f9d36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:12.107297 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed73efd-16ad-4ae4-b3d4-8f0624f9d36e admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.109420 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed73efd-16ad-4ae4-b3d4-8f0624f9d36e admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.110223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed73efd-16ad-4ae4-b3d4-8f0624f9d36e admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.112206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed73efd-16ad-4ae4-b3d4-8f0624f9d36e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:47:12.113727 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed73efd-16ad-4ae4-b3d4-8f0624f9d36e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:12.116416 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: WARNING py.warnings [None req-1ed73efd-16ad-4ae4-b3d4-8f0624f9d36e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 09 09:47:12.116416 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: warnings.warn(msg) May 09 09:47:12.116416 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: May 09 09:47:12.117683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed73efd-16ad-4ae4-b3d4-8f0624f9d36e admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:12.131780 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-a9db9f21-a90c-43b8-b083-addeb6d424f5 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:12.133393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a9db9f21-a90c-43b8-b083-addeb6d424f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:12.137342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 784/1547] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:47:12.138745 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41389925-677c-494f-bf7e-53f21f3a4365 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.139033 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41389925-677c-494f-bf7e-53f21f3a4365 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.139309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41389925-677c-494f-bf7e-53f21f3a4365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.143512 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 764/1548] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:12.145285 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba0493-440e-485a-9778-9c14644339a1 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.145787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba0493-440e-485a-9778-9c14644339a1 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.146322 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba0493-440e-485a-9778-9c14644339a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.164197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-41389925-677c-494f-bf7e-53f21f3a4365 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:12.164604 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-18ba0493-440e-485a-9778-9c14644339a1 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:12.165596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-41389925-677c-494f-bf7e-53f21f3a4365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:12.165793 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-18ba0493-440e-485a-9778-9c14644339a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:12.173830 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 765/1549] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:12.174253 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 785/1550] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:12.175484 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3bff39-6abb-46fc-9b18-c61cbd8b5952 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.175830 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3bff39-6abb-46fc-9b18-c61cbd8b5952 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.176164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3bff39-6abb-46fc-9b18-c61cbd8b5952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.177889 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba0493-440e-485a-9778-9c14644339a1 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.192107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba0493-440e-485a-9778-9c14644339a1 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:12.195258 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-7f3bff39-6abb-46fc-9b18-c61cbd8b5952 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:12.196619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-7f3bff39-6abb-46fc-9b18-c61cbd8b5952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:12.197130 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee2612-0294-4edd-a53f-9f70d45f57d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:12.197792 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee2612-0294-4edd-a53f-9f70d45f57d6 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.198066 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee2612-0294-4edd-a53f-9f70d45f57d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.198454 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee2612-0294-4edd-a53f-9f70d45f57d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.199161 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee2612-0294-4edd-a53f-9f70d45f57d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:12.200333 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: WARNING py.warnings [None req-c7ee2612-0294-4edd-a53f-9f70d45f57d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 09:47:12.200333 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: warnings.warn(msg) May 09 09:47:12.200333 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: May 09 09:47:12.201590 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee2612-0294-4edd-a53f-9f70d45f57d6 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:12.204498 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 786/1551] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:12.205295 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3bff39-6abb-46fc-9b18-c61cbd8b5952 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.214607 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3bff39-6abb-46fc-9b18-c61cbd8b5952 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:12.220250 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3c2bb-aad9-42ce-a44b-e6ccbbba369a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:12.221038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3c2bb-aad9-42ce-a44b-e6ccbbba369a admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.221312 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3c2bb-aad9-42ce-a44b-e6ccbbba369a admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.221581 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3c2bb-aad9-42ce-a44b-e6ccbbba369a admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.222307 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3c2bb-aad9-42ce-a44b-e6ccbbba369a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:47:12.222601 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3c2bb-aad9-42ce-a44b-e6ccbbba369a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:12.224193 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3c2bb-aad9-42ce-a44b-e6ccbbba369a admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:12.227949 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 766/1552] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:12 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:12.229267 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bfc5a-ec47-4c1f-8880-8a4b59338224 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.229553 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bfc5a-ec47-4c1f-8880-8a4b59338224 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.229824 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bfc5a-ec47-4c1f-8880-8a4b59338224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.240763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 787/1553] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:47:12.240763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3c2bb-aad9-42ce-a44b-e6ccbbba369a admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.247778 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-275bfc5a-ec47-4c1f-8880-8a4b59338224 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:12.248996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-275bfc5a-ec47-4c1f-8880-8a4b59338224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:12.249446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3c2bb-aad9-42ce-a44b-e6ccbbba369a admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:12.303336 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298f524-be66-4c73-93e8-2c695fcb8fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:12.304012 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298f524-be66-4c73-93e8-2c695fcb8fff admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.304289 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298f524-be66-4c73-93e8-2c695fcb8fff admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.304556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298f524-be66-4c73-93e8-2c695fcb8fff admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.305006 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 767/1554] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:12.305259 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298f524-be66-4c73-93e8-2c695fcb8fff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:12.305846 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bfc5a-ec47-4c1f-8880-8a4b59338224 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.307198 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298f524-be66-4c73-93e8-2c695fcb8fff admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:12.315214 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bfc5a-ec47-4c1f-8880-8a4b59338224 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:12.319746 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d62af-b2cc-425f-a516-8b49efe72a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:12.320395 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d62af-b2cc-425f-a516-8b49efe72a10 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.320699 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d62af-b2cc-425f-a516-8b49efe72a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.320973 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d62af-b2cc-425f-a516-8b49efe72a10 admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.321698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d62af-b2cc-425f-a516-8b49efe72a10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:47:12.321999 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d62af-b2cc-425f-a516-8b49efe72a10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:12.323692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d62af-b2cc-425f-a516-8b49efe72a10 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:12.334459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 788/1555] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:12 2022] POST /identity/v3/projects => generated 393 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:12.342532 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 768/1556] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:47:12.342708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137bc088-c1b0-4ed0-b56b-1e46e90a7e13 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.342708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137bc088-c1b0-4ed0-b56b-1e46e90a7e13 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.342708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137bc088-c1b0-4ed0-b56b-1e46e90a7e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.342991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f49b08-a5a0-439a-86fa-72ab8c6da21f None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.342991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f49b08-a5a0-439a-86fa-72ab8c6da21f None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.342991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f49b08-a5a0-439a-86fa-72ab8c6da21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.354605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-137bc088-c1b0-4ed0-b56b-1e46e90a7e13 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:12.354798 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-89f49b08-a5a0-439a-86fa-72ab8c6da21f None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:12.355796 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-137bc088-c1b0-4ed0-b56b-1e46e90a7e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:12.355921 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-89f49b08-a5a0-439a-86fa-72ab8c6da21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:12.410909 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 769/1557] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:12.411432 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 789/1558] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:12.412416 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137bc088-c1b0-4ed0-b56b-1e46e90a7e13 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.412907 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb1544-2b72-4a40-8db5-06e585e7435b None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.413198 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb1544-2b72-4a40-8db5-06e585e7435b None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.413470 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb1544-2b72-4a40-8db5-06e585e7435b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.427063 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137bc088-c1b0-4ed0-b56b-1e46e90a7e13 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:12.433372 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a25071-1c3b-44ff-bb0a-6f08ff89aa55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:12.434077 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a25071-1c3b-44ff-bb0a-6f08ff89aa55 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.434358 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a25071-1c3b-44ff-bb0a-6f08ff89aa55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.434631 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a25071-1c3b-44ff-bb0a-6f08ff89aa55 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.435337 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-dbeb1544-2b72-4a40-8db5-06e585e7435b None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:12.436876 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a25071-1c3b-44ff-bb0a-6f08ff89aa55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:12.437019 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-dbeb1544-2b72-4a40-8db5-06e585e7435b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:12.438018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: WARNING py.warnings [None req-d5a25071-1c3b-44ff-bb0a-6f08ff89aa55 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 09:47:12.438018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: warnings.warn(msg) May 09 09:47:12.438018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: May 09 09:47:12.440956 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a25071-1c3b-44ff-bb0a-6f08ff89aa55 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:12.444884 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 770/1559] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:12.445483 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb1544-2b72-4a40-8db5-06e585e7435b None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.458474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb1544-2b72-4a40-8db5-06e585e7435b None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:12.463326 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd5001-4400-48ff-b5db-32eacc1a0d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:12.464149 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd5001-4400-48ff-b5db-32eacc1a0d14 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.464496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd5001-4400-48ff-b5db-32eacc1a0d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.464864 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd5001-4400-48ff-b5db-32eacc1a0d14 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.465548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 790/1560] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:12 2022] POST /identity/v3/users => generated 291 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:12.469302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd5001-4400-48ff-b5db-32eacc1a0d14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:12.469425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67ffd7-c3f4-4861-a2b7-0e827618d85c None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.469425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67ffd7-c3f4-4861-a2b7-0e827618d85c None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.469425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67ffd7-c3f4-4861-a2b7-0e827618d85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.469686 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd5001-4400-48ff-b5db-32eacc1a0d14 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:12.485768 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-ac67ffd7-c3f4-4861-a2b7-0e827618d85c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:12.487376 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-ac67ffd7-c3f4-4861-a2b7-0e827618d85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:12.493718 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 771/1561] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:12 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:12.494571 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd5001-4400-48ff-b5db-32eacc1a0d14 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.501997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 791/1562] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:12.501997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67ffd7-c3f4-4861-a2b7-0e827618d85c None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.508186 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67ffd7-c3f4-4861-a2b7-0e827618d85c None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:12.559540 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd5001-4400-48ff-b5db-32eacc1a0d14 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:12.564195 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81cbf1-c002-4391-9bbf-1a2030bc94bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:12.564379 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50071f97-be90-4f11-8dd8-8530ebdf4270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:12.564887 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81cbf1-c002-4391-9bbf-1a2030bc94bd admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.565033 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50071f97-be90-4f11-8dd8-8530ebdf4270 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.565130 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81cbf1-c002-4391-9bbf-1a2030bc94bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.565298 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50071f97-be90-4f11-8dd8-8530ebdf4270 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.565455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81cbf1-c002-4391-9bbf-1a2030bc94bd admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.565571 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50071f97-be90-4f11-8dd8-8530ebdf4270 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.566082 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81cbf1-c002-4391-9bbf-1a2030bc94bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:12.566217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50071f97-be90-4f11-8dd8-8530ebdf4270 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:12.567968 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81cbf1-c002-4391-9bbf-1a2030bc94bd admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:12.568190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50071f97-be90-4f11-8dd8-8530ebdf4270 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:12.596534 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 772/1563] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:12 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:12.597473 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50071f97-be90-4f11-8dd8-8530ebdf4270 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.601595 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 792/1564] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:12 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:12.601595 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3151d-a11d-43e8-a778-d7a05501f75a None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.601595 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3151d-a11d-43e8-a778-d7a05501f75a None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.601595 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3151d-a11d-43e8-a778-d7a05501f75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.607492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50071f97-be90-4f11-8dd8-8530ebdf4270 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:12.619218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-47b3151d-a11d-43e8-a778-d7a05501f75a None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:12.620417 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-47b3151d-a11d-43e8-a778-d7a05501f75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:12.662274 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfa31a-6141-4b6c-b19d-2ec2b5dbcfac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:12.662855 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfa31a-6141-4b6c-b19d-2ec2b5dbcfac admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.663131 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfa31a-6141-4b6c-b19d-2ec2b5dbcfac admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.663398 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfa31a-6141-4b6c-b19d-2ec2b5dbcfac admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.664120 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfa31a-6141-4b6c-b19d-2ec2b5dbcfac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:47:12.664421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfa31a-6141-4b6c-b19d-2ec2b5dbcfac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:12.666065 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfa31a-6141-4b6c-b19d-2ec2b5dbcfac admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:12.675964 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 793/1565] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:12.676855 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3151d-a11d-43e8-a778-d7a05501f75a None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.677273 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 773/1566] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:47:12.677951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfa31a-6141-4b6c-b19d-2ec2b5dbcfac admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.689527 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfa31a-6141-4b6c-b19d-2ec2b5dbcfac admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:12.689888 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3151d-a11d-43e8-a778-d7a05501f75a None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:12.694409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c6e57-5e95-4d28-bdfa-dec1151c8b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:12.694558 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62199361-a541-4654-a867-f64b0a422dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:12.695061 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c6e57-5e95-4d28-bdfa-dec1151c8b18 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.695196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62199361-a541-4654-a867-f64b0a422dad admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.695338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c6e57-5e95-4d28-bdfa-dec1151c8b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.695474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62199361-a541-4654-a867-f64b0a422dad admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.695608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c6e57-5e95-4d28-bdfa-dec1151c8b18 admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.695889 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62199361-a541-4654-a867-f64b0a422dad admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.696335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c6e57-5e95-4d28-bdfa-dec1151c8b18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:47:12.696619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62199361-a541-4654-a867-f64b0a422dad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:12.696762 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c6e57-5e95-4d28-bdfa-dec1151c8b18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:12.698263 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c6e57-5e95-4d28-bdfa-dec1151c8b18 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:12.698375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62199361-a541-4654-a867-f64b0a422dad admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:12.708761 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 774/1567] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:12.716446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 794/1568] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:47:12.716617 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62199361-a541-4654-a867-f64b0a422dad admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.716749 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c6e57-5e95-4d28-bdfa-dec1151c8b18 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.720950 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62199361-a541-4654-a867-f64b0a422dad admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:12.725758 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62e5d0-6db9-4e39-97c1-f98ab3702d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:12.726404 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62e5d0-6db9-4e39-97c1-f98ab3702d66 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.726679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62e5d0-6db9-4e39-97c1-f98ab3702d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.726787 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c6e57-5e95-4d28-bdfa-dec1151c8b18 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:12.726949 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62e5d0-6db9-4e39-97c1-f98ab3702d66 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.727628 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62e5d0-6db9-4e39-97c1-f98ab3702d66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:12.729624 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62e5d0-6db9-4e39-97c1-f98ab3702d66 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:12.733322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6af3c7-042a-4e5b-8494-5fe129f07885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:12.734073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6af3c7-042a-4e5b-8494-5fe129f07885 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.734356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6af3c7-042a-4e5b-8494-5fe129f07885 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.734636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6af3c7-042a-4e5b-8494-5fe129f07885 admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.737041 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6af3c7-042a-4e5b-8494-5fe129f07885 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:47:12.737359 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6af3c7-042a-4e5b-8494-5fe129f07885 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:12.739676 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6af3c7-042a-4e5b-8494-5fe129f07885 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:12.754007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 795/1569] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:47:12.761037 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6af3c7-042a-4e5b-8494-5fe129f07885 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.766050 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 775/1570] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:12 2022] POST /identity/v3/users => generated 289 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:12.766289 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6af3c7-042a-4e5b-8494-5fe129f07885 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:12.767629 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62e5d0-6db9-4e39-97c1-f98ab3702d66 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.771813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ad59f-a3cf-45b2-a77d-85d77ba0e1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:12.772470 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ad59f-a3cf-45b2-a77d-85d77ba0e1ea admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.772774 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ad59f-a3cf-45b2-a77d-85d77ba0e1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.773047 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ad59f-a3cf-45b2-a77d-85d77ba0e1ea admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.773755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ad59f-a3cf-45b2-a77d-85d77ba0e1ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:12.775345 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ad59f-a3cf-45b2-a77d-85d77ba0e1ea admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:12.778898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62e5d0-6db9-4e39-97c1-f98ab3702d66 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:12.784006 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb1414-2de0-4e54-a7fd-464183d4c714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:12.784681 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb1414-2de0-4e54-a7fd-464183d4c714 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.784993 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb1414-2de0-4e54-a7fd-464183d4c714 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.785254 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb1414-2de0-4e54-a7fd-464183d4c714 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.785935 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 796/1571] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:12.786071 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb1414-2de0-4e54-a7fd-464183d4c714 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:12.786711 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ad59f-a3cf-45b2-a77d-85d77ba0e1ea admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.787938 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb1414-2de0-4e54-a7fd-464183d4c714 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:12.798321 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ad59f-a3cf-45b2-a77d-85d77ba0e1ea admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:12.803986 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c2331-55d7-4089-8635-787c311d9db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:12.804701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c2331-55d7-4089-8635-787c311d9db4 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.804977 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c2331-55d7-4089-8635-787c311d9db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.805244 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c2331-55d7-4089-8635-787c311d9db4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.805983 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c2331-55d7-4089-8635-787c311d9db4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:47:12.806283 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c2331-55d7-4089-8635-787c311d9db4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:12.807878 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c2331-55d7-4089-8635-787c311d9db4 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:12.809487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 776/1572] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:12 2022] POST /identity/v3/users => generated 288 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:12.810289 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb1414-2de0-4e54-a7fd-464183d4c714 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.819761 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb1414-2de0-4e54-a7fd-464183d4c714 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:12.829355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 797/1573] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:47:12.829355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c2331-55d7-4089-8635-787c311d9db4 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.830052 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c70ba3-7591-4af7-bef2-64860bc6311b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:12.831077 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c70ba3-7591-4af7-bef2-64860bc6311b admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.831522 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c70ba3-7591-4af7-bef2-64860bc6311b admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.831968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c70ba3-7591-4af7-bef2-64860bc6311b admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.833176 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c70ba3-7591-4af7-bef2-64860bc6311b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:12.836538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c70ba3-7591-4af7-bef2-64860bc6311b admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:12.844236 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c2331-55d7-4089-8635-787c311d9db4 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:12.850992 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b8797-d47c-4a26-8dca-d33096c91ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:12.851998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b8797-d47c-4a26-8dca-d33096c91ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.852395 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b8797-d47c-4a26-8dca-d33096c91ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.852875 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b8797-d47c-4a26-8dca-d33096c91ba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.854083 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073b8797-d47c-4a26-8dca-d33096c91ba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:12.856879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073b8797-d47c-4a26-8dca-d33096c91ba7 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:12.869617 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 798/1574] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:12.870812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b8797-d47c-4a26-8dca-d33096c91ba7 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.871005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 777/1575] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:12 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:12.871561 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c70ba3-7591-4af7-bef2-64860bc6311b admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.885246 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b8797-d47c-4a26-8dca-d33096c91ba7 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:12.933187 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c70ba3-7591-4af7-bef2-64860bc6311b admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:12.937639 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f1d2b8-8d59-4783-8cbd-0080a88715b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:12.938315 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1d2b8-8d59-4783-8cbd-0080a88715b0 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.938590 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1d2b8-8d59-4783-8cbd-0080a88715b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.938855 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1d2b8-8d59-4783-8cbd-0080a88715b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.938959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73e5fb-0a1d-4a5b-98d5-dd54a14b3e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:12.939533 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f1d2b8-8d59-4783-8cbd-0080a88715b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:12.939633 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73e5fb-0a1d-4a5b-98d5-dd54a14b3e39 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:12.939809 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73e5fb-0a1d-4a5b-98d5-dd54a14b3e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:12.940084 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73e5fb-0a1d-4a5b-98d5-dd54a14b3e39 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:12.940789 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73e5fb-0a1d-4a5b-98d5-dd54a14b3e39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:12.941456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f1d2b8-8d59-4783-8cbd-0080a88715b0 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:12.942776 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73e5fb-0a1d-4a5b-98d5-dd54a14b3e39 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:12.966785 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 778/1576] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:12 2022] POST /identity/v3/projects => generated 405 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:12.967596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f1d2b8-8d59-4783-8cbd-0080a88715b0 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:12.979363 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 799/1577] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:12 2022] POST /identity/v3/projects => generated 389 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:12.979363 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73e5fb-0a1d-4a5b-98d5-dd54a14b3e39 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.026914 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f1d2b8-8d59-4783-8cbd-0080a88715b0 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.031316 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b447f-a626-4c3d-b5f4-eeb0a0ac19ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.032180 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b447f-a626-4c3d-b5f4-eeb0a0ac19ad admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.032760 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b447f-a626-4c3d-b5f4-eeb0a0ac19ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.032760 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b447f-a626-4c3d-b5f4-eeb0a0ac19ad admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.033263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b447f-a626-4c3d-b5f4-eeb0a0ac19ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.034880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b447f-a626-4c3d-b5f4-eeb0a0ac19ad admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.036080 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73e5fb-0a1d-4a5b-98d5-dd54a14b3e39 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.040844 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e483e1-eea5-4448-a308-b47a3a6734e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.041503 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e483e1-eea5-4448-a308-b47a3a6734e3 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.041782 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e483e1-eea5-4448-a308-b47a3a6734e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.042054 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e483e1-eea5-4448-a308-b47a3a6734e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.042783 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e483e1-eea5-4448-a308-b47a3a6734e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.044324 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 779/1578] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:13.045135 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b447f-a626-4c3d-b5f4-eeb0a0ac19ad admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.047771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e483e1-eea5-4448-a308-b47a3a6734e3 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.054373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b447f-a626-4c3d-b5f4-eeb0a0ac19ad admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.058746 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97e6c3-f472-4d21-9a7b-0da0cd27994f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.059385 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97e6c3-f472-4d21-9a7b-0da0cd27994f admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.059660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97e6c3-f472-4d21-9a7b-0da0cd27994f admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.059928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97e6c3-f472-4d21-9a7b-0da0cd27994f admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.060647 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97e6c3-f472-4d21-9a7b-0da0cd27994f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.062144 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 800/1579] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:13.062274 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97e6c3-f472-4d21-9a7b-0da0cd27994f admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.063093 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e483e1-eea5-4448-a308-b47a3a6734e3 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.075420 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 780/1580] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:13.075420 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97e6c3-f472-4d21-9a7b-0da0cd27994f admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.075726 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e483e1-eea5-4448-a308-b47a3a6734e3 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.080914 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3e901-14a8-41ec-9ca8-7494f822a013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.081573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3e901-14a8-41ec-9ca8-7494f822a013 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.081846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3e901-14a8-41ec-9ca8-7494f822a013 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.082111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3e901-14a8-41ec-9ca8-7494f822a013 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.082465 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97e6c3-f472-4d21-9a7b-0da0cd27994f admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.082840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3e901-14a8-41ec-9ca8-7494f822a013 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.084810 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3e901-14a8-41ec-9ca8-7494f822a013 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.087129 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0a285-49f9-4137-ae57-6755636d820c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.087961 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0a285-49f9-4137-ae57-6755636d820c admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.088233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0a285-49f9-4137-ae57-6755636d820c admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.088497 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0a285-49f9-4137-ae57-6755636d820c admin admin] PATH_INFO: `/v3/projects/23b8170c316f43eaa233c612251d499a/users/7e95fce19921454a99e6071423db7932/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.103497 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0a285-49f9-4137-ae57-6755636d820c admin admin] RBAC: Authorizing `identity:create_grant(project_id=23b8170c316f43eaa233c612251d499a, user_id=7e95fce19921454a99e6071423db7932, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.104583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: WARNING py.warnings [None req-55b0a285-49f9-4137-ae57-6755636d820c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 09:47:13.104583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: warnings.warn(msg) May 09 09:47:13.104583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: May 09 09:47:13.106568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0a285-49f9-4137-ae57-6755636d820c admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.112044 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 801/1581] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:13 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:13.112905 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3e901-14a8-41ec-9ca8-7494f822a013 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.121167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 781/1582] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:13 2022] PUT /identity/v3/projects/23b8170c316f43eaa233c612251d499a/users/7e95fce19921454a99e6071423db7932/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:13.122242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0a285-49f9-4137-ae57-6755636d820c admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.171897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3e901-14a8-41ec-9ca8-7494f822a013 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.180818 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0a285-49f9-4137-ae57-6755636d820c admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.185342 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942dedb-34b3-4d9b-94fe-2dfb5a7f1447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.186085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942dedb-34b3-4d9b-94fe-2dfb5a7f1447 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.186366 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942dedb-34b3-4d9b-94fe-2dfb5a7f1447 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.186630 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942dedb-34b3-4d9b-94fe-2dfb5a7f1447 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.187304 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942dedb-34b3-4d9b-94fe-2dfb5a7f1447 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.189301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942dedb-34b3-4d9b-94fe-2dfb5a7f1447 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.208648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 782/1583] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:13 2022] POST /identity/v3/users => generated 303 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:13.209507 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942dedb-34b3-4d9b-94fe-2dfb5a7f1447 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.218834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942dedb-34b3-4d9b-94fe-2dfb5a7f1447 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.223429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529e997-1904-488c-b0b0-e9fb5f328358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.224063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529e997-1904-488c-b0b0-e9fb5f328358 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.224338 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529e997-1904-488c-b0b0-e9fb5f328358 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.224607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529e997-1904-488c-b0b0-e9fb5f328358 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.225297 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529e997-1904-488c-b0b0-e9fb5f328358 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.227195 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529e997-1904-488c-b0b0-e9fb5f328358 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.230194 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9e1ab-c36b-4176-a10b-acb498cb1317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.230889 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9e1ab-c36b-4176-a10b-acb498cb1317 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.231160 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9e1ab-c36b-4176-a10b-acb498cb1317 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.231457 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9e1ab-c36b-4176-a10b-acb498cb1317 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.232184 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd9e1ab-c36b-4176-a10b-acb498cb1317 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.234141 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd9e1ab-c36b-4176-a10b-acb498cb1317 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.247660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 783/1584] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:13 2022] POST /identity/v3/users => generated 295 bytes in 38 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:13.248463 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529e997-1904-488c-b0b0-e9fb5f328358 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.254427 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 802/1585] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:13 2022] POST /identity/v3/users => generated 287 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:13.257648 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9e1ab-c36b-4176-a10b-acb498cb1317 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.258506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529e997-1904-488c-b0b0-e9fb5f328358 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.263019 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd970d9c-e19c-43ea-a690-e45558828733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.263669 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd970d9c-e19c-43ea-a690-e45558828733 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.263949 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd970d9c-e19c-43ea-a690-e45558828733 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.264220 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd970d9c-e19c-43ea-a690-e45558828733 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.264974 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd970d9c-e19c-43ea-a690-e45558828733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.266589 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd970d9c-e19c-43ea-a690-e45558828733 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.267193 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9e1ab-c36b-4176-a10b-acb498cb1317 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.271596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d5acc-4b92-466d-9180-58776693d1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.272483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d5acc-4b92-466d-9180-58776693d1dd admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.272796 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d5acc-4b92-466d-9180-58776693d1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.273063 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d5acc-4b92-466d-9180-58776693d1dd admin admin] PATH_INFO: `/v3/projects/0511fef4fd9f4bd09a3b1867e8556a06/users/e58c14dca7da4e6495f7e08f2bb127e2/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.277053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 784/1586] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:13.280404 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd970d9c-e19c-43ea-a690-e45558828733 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.287117 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd970d9c-e19c-43ea-a690-e45558828733 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.287383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d5acc-4b92-466d-9180-58776693d1dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0511fef4fd9f4bd09a3b1867e8556a06, user_id=e58c14dca7da4e6495f7e08f2bb127e2, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.290079 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d5acc-4b92-466d-9180-58776693d1dd admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.292221 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d35964-1e8c-4527-b081-b08752695324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.293034 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d35964-1e8c-4527-b081-b08752695324 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.293314 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d35964-1e8c-4527-b081-b08752695324 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.293619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d35964-1e8c-4527-b081-b08752695324 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.294420 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d35964-1e8c-4527-b081-b08752695324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.296682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d35964-1e8c-4527-b081-b08752695324 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.303321 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 803/1587] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:13 2022] PUT /identity/v3/projects/0511fef4fd9f4bd09a3b1867e8556a06/users/e58c14dca7da4e6495f7e08f2bb127e2/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:13.304076 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d5acc-4b92-466d-9180-58776693d1dd admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.313964 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 785/1588] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:13.315246 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b52d29-ea57-45ea-9359-c5ae7d33f036 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.315529 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b52d29-ea57-45ea-9359-c5ae7d33f036 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.315804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b52d29-ea57-45ea-9359-c5ae7d33f036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.341019 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-e6b52d29-ea57-45ea-9359-c5ae7d33f036 None None] MFA Rules not processed for user `7e95fce19921454a99e6071423db7932`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:13.342234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e6b52d29-ea57-45ea-9359-c5ae7d33f036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:13.363949 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d5acc-4b92-466d-9180-58776693d1dd admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.368771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf519e-79ad-4cf1-95a5-c79ac479841a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.369432 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf519e-79ad-4cf1-95a5-c79ac479841a admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.369794 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf519e-79ad-4cf1-95a5-c79ac479841a admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.370083 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf519e-79ad-4cf1-95a5-c79ac479841a admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.370778 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf519e-79ad-4cf1-95a5-c79ac479841a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.372804 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf519e-79ad-4cf1-95a5-c79ac479841a admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.392895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 804/1589] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:13 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:13.393795 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf519e-79ad-4cf1-95a5-c79ac479841a admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.404540 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 786/1590] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:13.405427 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf519e-79ad-4cf1-95a5-c79ac479841a admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.405564 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b52d29-ea57-45ea-9359-c5ae7d33f036 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.409954 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c87663-9ed6-412c-962c-70d982b7a941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.410593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c87663-9ed6-412c-962c-70d982b7a941 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.410894 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c87663-9ed6-412c-962c-70d982b7a941 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.411162 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c87663-9ed6-412c-962c-70d982b7a941 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.411914 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c87663-9ed6-412c-962c-70d982b7a941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.413763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c87663-9ed6-412c-962c-70d982b7a941 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.417621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b52d29-ea57-45ea-9359-c5ae7d33f036 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.422518 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a21885-45cc-43cf-92e7-6f92496b2251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.423163 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a21885-45cc-43cf-92e7-6f92496b2251 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.423441 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a21885-45cc-43cf-92e7-6f92496b2251 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.423712 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a21885-45cc-43cf-92e7-6f92496b2251 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.423837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 805/1591] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:13.424435 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a21885-45cc-43cf-92e7-6f92496b2251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.424576 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c87663-9ed6-412c-962c-70d982b7a941 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.426082 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a21885-45cc-43cf-92e7-6f92496b2251 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.435526 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c87663-9ed6-412c-962c-70d982b7a941 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.436130 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 787/1592] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:13.436939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a21885-45cc-43cf-92e7-6f92496b2251 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.440447 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b47a011-1b52-4327-b1e8-b5670499be41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.441115 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47a011-1b52-4327-b1e8-b5670499be41 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.441390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47a011-1b52-4327-b1e8-b5670499be41 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.441659 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47a011-1b52-4327-b1e8-b5670499be41 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.442356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b47a011-1b52-4327-b1e8-b5670499be41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.443933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b47a011-1b52-4327-b1e8-b5670499be41 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.446303 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a21885-45cc-43cf-92e7-6f92496b2251 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.450867 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc36218-a6eb-436b-ada7-2bf24731885c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.451702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc36218-a6eb-436b-ada7-2bf24731885c admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.451990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc36218-a6eb-436b-ada7-2bf24731885c admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.452244 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc36218-a6eb-436b-ada7-2bf24731885c admin admin] PATH_INFO: `/v3/projects/16fcb7e90638419cbd62a6f77511ecd0/users/e5dc66d5bb814351a94facdc743c2296/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.454325 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 806/1593] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:13.455464 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df947f8d-fd2d-4e0c-8dee-ba141336c583 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.455748 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df947f8d-fd2d-4e0c-8dee-ba141336c583 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.456015 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df947f8d-fd2d-4e0c-8dee-ba141336c583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.467500 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc36218-a6eb-436b-ada7-2bf24731885c admin admin] RBAC: Authorizing `identity:create_grant(project_id=16fcb7e90638419cbd62a6f77511ecd0, user_id=e5dc66d5bb814351a94facdc743c2296, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.470266 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc36218-a6eb-436b-ada7-2bf24731885c admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.472403 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-df947f8d-fd2d-4e0c-8dee-ba141336c583 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:13.473650 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-df947f8d-fd2d-4e0c-8dee-ba141336c583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:13.482378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 807/1594] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:13.485676 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df947f8d-fd2d-4e0c-8dee-ba141336c583 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.485781 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 788/1595] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:13 2022] PUT /identity/v3/projects/16fcb7e90638419cbd62a6f77511ecd0/users/e5dc66d5bb814351a94facdc743c2296/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:13.485991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc36218-a6eb-436b-ada7-2bf24731885c admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.506149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df947f8d-fd2d-4e0c-8dee-ba141336c583 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.547108 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc36218-a6eb-436b-ada7-2bf24731885c admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.551782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f05e2-afcf-4ab7-aed6-1f850729ac4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.552444 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f05e2-afcf-4ab7-aed6-1f850729ac4e admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.552746 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f05e2-afcf-4ab7-aed6-1f850729ac4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.553016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f05e2-afcf-4ab7-aed6-1f850729ac4e admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.553741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f05e2-afcf-4ab7-aed6-1f850729ac4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.555321 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f05e2-afcf-4ab7-aed6-1f850729ac4e admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.562497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df5347-3c6a-483e-9a47-6f5558aae527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.563472 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df5347-3c6a-483e-9a47-6f5558aae527 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.563767 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df5347-3c6a-483e-9a47-6f5558aae527 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.564033 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df5347-3c6a-483e-9a47-6f5558aae527 admin admin] PATH_INFO: `/v3/projects/c90b9a64e6a8464b9cde8bb967c4872d/users/f89c5ab1020947a9bece03af4c237480/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.565403 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 789/1596] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:13.566449 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f05e2-afcf-4ab7-aed6-1f850729ac4e admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.576074 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f05e2-afcf-4ab7-aed6-1f850729ac4e admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.578238 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df5347-3c6a-483e-9a47-6f5558aae527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c90b9a64e6a8464b9cde8bb967c4872d, user_id=f89c5ab1020947a9bece03af4c237480, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.580545 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840801d-267f-4649-acae-f21fca3c73a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.580959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df5347-3c6a-483e-9a47-6f5558aae527 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.581226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840801d-267f-4649-acae-f21fca3c73a5 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.581515 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840801d-267f-4649-acae-f21fca3c73a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.581780 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840801d-267f-4649-acae-f21fca3c73a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.582490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840801d-267f-4649-acae-f21fca3c73a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.584342 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840801d-267f-4649-acae-f21fca3c73a5 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.594432 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 808/1597] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:13 2022] PUT /identity/v3/projects/c90b9a64e6a8464b9cde8bb967c4872d/users/f89c5ab1020947a9bece03af4c237480/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:13.595277 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df5347-3c6a-483e-9a47-6f5558aae527 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.619104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 790/1598] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:13 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:13.619104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840801d-267f-4649-acae-f21fca3c73a5 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.660162 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df5347-3c6a-483e-9a47-6f5558aae527 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.687725 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840801d-267f-4649-acae-f21fca3c73a5 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.693081 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b667c3-aff9-415d-a520-15399eabd51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.693749 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b667c3-aff9-415d-a520-15399eabd51f admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.694027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b667c3-aff9-415d-a520-15399eabd51f admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.694297 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b667c3-aff9-415d-a520-15399eabd51f admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.695025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b667c3-aff9-415d-a520-15399eabd51f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.696644 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b667c3-aff9-415d-a520-15399eabd51f admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.706866 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 791/1599] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:13.707708 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b667c3-aff9-415d-a520-15399eabd51f admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.715504 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cf113-1c73-4f79-81b2-deb26b3cb27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.716189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cf113-1c73-4f79-81b2-deb26b3cb27f admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.716492 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cf113-1c73-4f79-81b2-deb26b3cb27f admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.716861 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cf113-1c73-4f79-81b2-deb26b3cb27f admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.717528 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b667c3-aff9-415d-a520-15399eabd51f admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.717651 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542cf113-1c73-4f79-81b2-deb26b3cb27f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.719641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542cf113-1c73-4f79-81b2-deb26b3cb27f admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.722380 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3bfbb-c876-4a45-a887-0a552e6038a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.722952 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3bfbb-c876-4a45-a887-0a552e6038a6 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.723283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3bfbb-c876-4a45-a887-0a552e6038a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.723475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3bfbb-c876-4a45-a887-0a552e6038a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.724222 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3bfbb-c876-4a45-a887-0a552e6038a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.725835 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3bfbb-c876-4a45-a887-0a552e6038a6 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.729315 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 809/1600] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:13.730597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b3b63-7b15-472b-9dc1-b6e0d4f3e470 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.730883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b3b63-7b15-472b-9dc1-b6e0d4f3e470 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.731152 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b3b63-7b15-472b-9dc1-b6e0d4f3e470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.736974 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 792/1601] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:13.737785 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3bfbb-c876-4a45-a887-0a552e6038a6 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.757160 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-148b3b63-7b15-472b-9dc1-b6e0d4f3e470 None None] MFA Rules not processed for user `e5dc66d5bb814351a94facdc743c2296`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:13.758380 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-148b3b63-7b15-472b-9dc1-b6e0d4f3e470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:13.792411 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3bfbb-c876-4a45-a887-0a552e6038a6 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.795991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41706f9-2cd6-4d6d-b390-ee9ab6ba3476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.796631 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41706f9-2cd6-4d6d-b390-ee9ab6ba3476 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.796929 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41706f9-2cd6-4d6d-b390-ee9ab6ba3476 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.797214 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41706f9-2cd6-4d6d-b390-ee9ab6ba3476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.806490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41706f9-2cd6-4d6d-b390-ee9ab6ba3476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.807984 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41706f9-2cd6-4d6d-b390-ee9ab6ba3476 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.817441 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 793/1602] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:13.818278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41706f9-2cd6-4d6d-b390-ee9ab6ba3476 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.822863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 810/1603] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:13.822863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4b6e1-475c-42c4-a36c-f2d4deb444b5 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.822863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4b6e1-475c-42c4-a36c-f2d4deb444b5 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.822863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4b6e1-475c-42c4-a36c-f2d4deb444b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.828612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41706f9-2cd6-4d6d-b390-ee9ab6ba3476 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.833422 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ba0e5-e511-4c48-bcc0-6f608d1a7fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.834082 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ba0e5-e511-4c48-bcc0-6f608d1a7fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.834362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ba0e5-e511-4c48-bcc0-6f608d1a7fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.834635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ba0e5-e511-4c48-bcc0-6f608d1a7fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.835394 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ba0e5-e511-4c48-bcc0-6f608d1a7fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.837045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ba0e5-e511-4c48-bcc0-6f608d1a7fd4 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.840787 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-3ee4b6e1-475c-42c4-a36c-f2d4deb444b5 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:13.842645 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-3ee4b6e1-475c-42c4-a36c-f2d4deb444b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:13.847560 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 794/1604] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:13.848201 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ba0e5-e511-4c48-bcc0-6f608d1a7fd4 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.854054 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 811/1605] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:13.854054 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4b6e1-475c-42c4-a36c-f2d4deb444b5 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.862981 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4b6e1-475c-42c4-a36c-f2d4deb444b5 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.864545 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ba0e5-e511-4c48-bcc0-6f608d1a7fd4 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.868986 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751abcc0-fe07-4771-90fd-fa40c1a8e08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.869383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089eb702-7cc5-49c6-847c-dc972de0c824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.870043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089eb702-7cc5-49c6-847c-dc972de0c824 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.870318 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089eb702-7cc5-49c6-847c-dc972de0c824 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.870586 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089eb702-7cc5-49c6-847c-dc972de0c824 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.870781 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751abcc0-fe07-4771-90fd-fa40c1a8e08c admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.871195 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751abcc0-fe07-4771-90fd-fa40c1a8e08c admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.871325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089eb702-7cc5-49c6-847c-dc972de0c824 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.871723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751abcc0-fe07-4771-90fd-fa40c1a8e08c admin admin] PATH_INFO: `/v3/projects/8c91ae2e41a9487a9ae5fd4bf1b6d4f8/users/76dc34045544496ab030864c5a0ffbff/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.873236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089eb702-7cc5-49c6-847c-dc972de0c824 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.889291 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751abcc0-fe07-4771-90fd-fa40c1a8e08c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c91ae2e41a9487a9ae5fd4bf1b6d4f8, user_id=76dc34045544496ab030864c5a0ffbff, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.892058 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751abcc0-fe07-4771-90fd-fa40c1a8e08c admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:13.895686 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 795/1606] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:13 2022] POST /identity/v3/users => generated 297 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:13.896680 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089eb702-7cc5-49c6-847c-dc972de0c824 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.907730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 812/1607] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:13 2022] PUT /identity/v3/projects/8c91ae2e41a9487a9ae5fd4bf1b6d4f8/users/76dc34045544496ab030864c5a0ffbff/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:13.908634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089eb702-7cc5-49c6-847c-dc972de0c824 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.908799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751abcc0-fe07-4771-90fd-fa40c1a8e08c admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:13.970097 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a736088-bebc-4d66-b2e7-ee083812dc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.971440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a736088-bebc-4d66-b2e7-ee083812dc97 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.971440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a736088-bebc-4d66-b2e7-ee083812dc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.971681 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a736088-bebc-4d66-b2e7-ee083812dc97 admin admin] PATH_INFO: `/v3/projects/e9aaa358e62342b1a17858f9c1356656/users/8069b466c6354d6d96bfd1bdb5f7710b/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.975139 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751abcc0-fe07-4771-90fd-fa40c1a8e08c admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:13.979951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d8f1f-3cc3-4b7c-9fe6-23651c0b0384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:13.981003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d8f1f-3cc3-4b7c-9fe6-23651c0b0384 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:13.981346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d8f1f-3cc3-4b7c-9fe6-23651c0b0384 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:13.981549 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d8f1f-3cc3-4b7c-9fe6-23651c0b0384 admin admin] PATH_INFO: `/v3/projects/ccaa0efe9f4d44d8899fabe0d5ed3d86/users/31fb080fb94046f8bc64575c367f5ec8/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:13.987039 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a736088-bebc-4d66-b2e7-ee083812dc97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9aaa358e62342b1a17858f9c1356656, user_id=8069b466c6354d6d96bfd1bdb5f7710b, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:13.989788 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a736088-bebc-4d66-b2e7-ee083812dc97 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:14.000537 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d8f1f-3cc3-4b7c-9fe6-23651c0b0384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccaa0efe9f4d44d8899fabe0d5ed3d86, user_id=31fb080fb94046f8bc64575c367f5ec8, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:14.002195 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d8f1f-3cc3-4b7c-9fe6-23651c0b0384 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:14.006751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 796/1608] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:13 2022] PUT /identity/v3/projects/e9aaa358e62342b1a17858f9c1356656/users/8069b466c6354d6d96bfd1bdb5f7710b/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:14.007407 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a736088-bebc-4d66-b2e7-ee083812dc97 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:14.027815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 813/1609] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:13 2022] PUT /identity/v3/projects/ccaa0efe9f4d44d8899fabe0d5ed3d86/users/31fb080fb94046f8bc64575c367f5ec8/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:14.027815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d8f1f-3cc3-4b7c-9fe6-23651c0b0384 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:14.076269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a736088-bebc-4d66-b2e7-ee083812dc97 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:14.081066 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047213f-af40-45f1-8bc5-f43b5072081f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:14.081732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047213f-af40-45f1-8bc5-f43b5072081f admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:14.082379 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047213f-af40-45f1-8bc5-f43b5072081f admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:14.082379 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047213f-af40-45f1-8bc5-f43b5072081f admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:14.082949 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9047213f-af40-45f1-8bc5-f43b5072081f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:14.084957 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9047213f-af40-45f1-8bc5-f43b5072081f admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:14.085254 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d8f1f-3cc3-4b7c-9fe6-23651c0b0384 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:14.092263 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410e074-5631-400b-b2f8-32db0088dca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:14.095465 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410e074-5631-400b-b2f8-32db0088dca0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:14.095465 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410e074-5631-400b-b2f8-32db0088dca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:14.095465 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410e074-5631-400b-b2f8-32db0088dca0 admin admin] PATH_INFO: `/v3/projects/153cae04bb6b46d0a52c15411f0c4720/users/40cb55020f07488c9c344541d895ec1c/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:14.112801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6410e074-5631-400b-b2f8-32db0088dca0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=153cae04bb6b46d0a52c15411f0c4720, user_id=40cb55020f07488c9c344541d895ec1c, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:14.115531 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 797/1610] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:14 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:14.115745 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6410e074-5631-400b-b2f8-32db0088dca0 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:14.116427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047213f-af40-45f1-8bc5-f43b5072081f admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:14.118670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-9047213f-af40-45f1-8bc5-f43b5072081f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:14.130456 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 814/1611] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:14 2022] PUT /identity/v3/projects/153cae04bb6b46d0a52c15411f0c4720/users/40cb55020f07488c9c344541d895ec1c/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:14.131772 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410e074-5631-400b-b2f8-32db0088dca0 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:14.181352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047213f-af40-45f1-8bc5-f43b5072081f admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:14.204054 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410e074-5631-400b-b2f8-32db0088dca0 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:14.209316 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8913c-2851-4492-a194-79a9f970f14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:14.209990 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8913c-2851-4492-a194-79a9f970f14f admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:14.210268 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8913c-2851-4492-a194-79a9f970f14f admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:14.210971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8913c-2851-4492-a194-79a9f970f14f admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:14.211782 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a8913c-2851-4492-a194-79a9f970f14f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:14.213383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a8913c-2851-4492-a194-79a9f970f14f admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:14.227239 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 815/1612] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:14.228631 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ed27e-1b5d-4d85-9718-6439a81f2ece None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:14.228940 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ed27e-1b5d-4d85-9718-6439a81f2ece None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:14.229215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ed27e-1b5d-4d85-9718-6439a81f2ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:14.237597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2ce43-1e89-4600-8277-a9edd5dfd592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:14.238266 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2ce43-1e89-4600-8277-a9edd5dfd592 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:14.238544 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2ce43-1e89-4600-8277-a9edd5dfd592 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:14.238816 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2ce43-1e89-4600-8277-a9edd5dfd592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:14.248993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-d90ed27e-1b5d-4d85-9718-6439a81f2ece None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:14.250548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-d90ed27e-1b5d-4d85-9718-6439a81f2ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:14.252907 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2ce43-1e89-4600-8277-a9edd5dfd592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:14.255618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2ce43-1e89-4600-8277-a9edd5dfd592 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:14.261352 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 816/1613] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:14.262909 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b7ecb-7552-4575-80cc-7c8670087cfd None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:14.263512 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b7ecb-7552-4575-80cc-7c8670087cfd None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:14.264004 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b7ecb-7552-4575-80cc-7c8670087cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:14.272519 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 798/1614] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:14.274458 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654526af-33d3-4fa8-8ecf-594f9e3500cb None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:14.274742 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654526af-33d3-4fa8-8ecf-594f9e3500cb None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:14.275054 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654526af-33d3-4fa8-8ecf-594f9e3500cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:14.299131 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-156b7ecb-7552-4575-80cc-7c8670087cfd None None] MFA Rules not processed for user `8069b466c6354d6d96bfd1bdb5f7710b`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:14.300364 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-156b7ecb-7552-4575-80cc-7c8670087cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:14.304742 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-654526af-33d3-4fa8-8ecf-594f9e3500cb None None] MFA Rules not processed for user `31fb080fb94046f8bc64575c367f5ec8`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:14.306004 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-654526af-33d3-4fa8-8ecf-594f9e3500cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:14.373629 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 817/1615] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:14 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:14.373629 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b7ecb-7552-4575-80cc-7c8670087cfd None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:14.376060 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 799/1616] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:14.385789 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79faa74-1fb4-4bf3-8af7-a0ad1c3cc08f None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:14.386525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79faa74-1fb4-4bf3-8af7-a0ad1c3cc08f None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:14.387000 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79faa74-1fb4-4bf3-8af7-a0ad1c3cc08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:14.402564 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b7ecb-7552-4575-80cc-7c8670087cfd None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:14.408324 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc025a-cbaa-47c6-bef6-22dbac4a5f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:14.409108 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc025a-cbaa-47c6-bef6-22dbac4a5f09 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:14.409387 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc025a-cbaa-47c6-bef6-22dbac4a5f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:14.409700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc025a-cbaa-47c6-bef6-22dbac4a5f09 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:14.410396 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfc025a-cbaa-47c6-bef6-22dbac4a5f09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:14.412805 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfc025a-cbaa-47c6-bef6-22dbac4a5f09 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:14.413712 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-c79faa74-1fb4-4bf3-8af7-a0ad1c3cc08f None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:14.415027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-c79faa74-1fb4-4bf3-8af7-a0ad1c3cc08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:14.424536 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 800/1617] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:14.430849 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79faa74-1fb4-4bf3-8af7-a0ad1c3cc08f None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:14.435583 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 818/1618] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:14 2022] POST /identity/v3/users => generated 295 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:14.437683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc025a-cbaa-47c6-bef6-22dbac4a5f09 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:14.445075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79faa74-1fb4-4bf3-8af7-a0ad1c3cc08f None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:14.448602 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc025a-cbaa-47c6-bef6-22dbac4a5f09 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:14.450521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311738e-b8b7-4365-8f14-9d4ffb11fcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:14.452391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0f18a-f776-4b57-863d-3b0605a9bc6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:14.452680 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311738e-b8b7-4365-8f14-9d4ffb11fcb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:14.453037 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311738e-b8b7-4365-8f14-9d4ffb11fcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:14.453269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0f18a-f776-4b57-863d-3b0605a9bc6b service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:14.453503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311738e-b8b7-4365-8f14-9d4ffb11fcb9 admin admin] PATH_INFO: `/v3/projects/b5b4118142524da281c8ba137e833456/users/843391e07bc0400eb41661ea0b2b1963/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:14.453784 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0f18a-f776-4b57-863d-3b0605a9bc6b service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:14.454158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0f18a-f776-4b57-863d-3b0605a9bc6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:14.464651 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0f18a-f776-4b57-863d-3b0605a9bc6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:14.466389 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0f18a-f776-4b57-863d-3b0605a9bc6b service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:14.469381 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311738e-b8b7-4365-8f14-9d4ffb11fcb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5b4118142524da281c8ba137e833456, user_id=843391e07bc0400eb41661ea0b2b1963, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:14.472453 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311738e-b8b7-4365-8f14-9d4ffb11fcb9 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:14.477505 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 819/1619] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:14.479450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0f18a-f776-4b57-863d-3b0605a9bc6b service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:14.491110 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0f18a-f776-4b57-863d-3b0605a9bc6b service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:14.491323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 801/1620] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:14 2022] PUT /identity/v3/projects/b5b4118142524da281c8ba137e833456/users/843391e07bc0400eb41661ea0b2b1963/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:14.492986 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df3f10-393d-42fe-a7a4-8a48fd887933 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:14.493554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df3f10-393d-42fe-a7a4-8a48fd887933 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:14.493836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df3f10-393d-42fe-a7a4-8a48fd887933 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:14.495299 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 802/1621] 10.222.0.26 () {54 vars in 961 bytes} [Mon May 9 09:47:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:47:14.496163 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df3f10-393d-42fe-a7a4-8a48fd887933 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:14.581795 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab9029-fb34-4f0a-9ef5-8d0ad41cac28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:14.582739 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab9029-fb34-4f0a-9ef5-8d0ad41cac28 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:14.583282 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab9029-fb34-4f0a-9ef5-8d0ad41cac28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:14.584068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab9029-fb34-4f0a-9ef5-8d0ad41cac28 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:14.584936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab9029-fb34-4f0a-9ef5-8d0ad41cac28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:14.587841 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab9029-fb34-4f0a-9ef5-8d0ad41cac28 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:14.594258 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df3f10-393d-42fe-a7a4-8a48fd887933 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:14.598582 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d2b2d-49ba-4f82-8c7a-1a275ba2a13e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:14.599321 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d2b2d-49ba-4f82-8c7a-1a275ba2a13e service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:14.599921 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d2b2d-49ba-4f82-8c7a-1a275ba2a13e service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:14.599921 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d2b2d-49ba-4f82-8c7a-1a275ba2a13e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:14.614699 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46d2b2d-49ba-4f82-8c7a-1a275ba2a13e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:14.616409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46d2b2d-49ba-4f82-8c7a-1a275ba2a13e service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:14.630157 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 803/1622] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:14.631267 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d2b2d-49ba-4f82-8c7a-1a275ba2a13e service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:14.631984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 820/1623] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:14 2022] POST /identity/v3/projects => generated 403 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:14.634582 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab9029-fb34-4f0a-9ef5-8d0ad41cac28 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:14.642380 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d2b2d-49ba-4f82-8c7a-1a275ba2a13e service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:14.720178 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbec301-129a-4d10-9787-d7488e73ef73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:14.721180 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbec301-129a-4d10-9787-d7488e73ef73 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:14.721562 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbec301-129a-4d10-9787-d7488e73ef73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:14.721974 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbec301-129a-4d10-9787-d7488e73ef73 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:14.722886 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbec301-129a-4d10-9787-d7488e73ef73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:14.725164 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbec301-129a-4d10-9787-d7488e73ef73 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:14.739518 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 804/1624] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:14.741113 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1e128-ae24-48e8-8c40-3e68f3c5eae8 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:14.741406 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1e128-ae24-48e8-8c40-3e68f3c5eae8 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:14.741682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1e128-ae24-48e8-8c40-3e68f3c5eae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:14.751201 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab9029-fb34-4f0a-9ef5-8d0ad41cac28 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:14.763851 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-12d1e128-ae24-48e8-8c40-3e68f3c5eae8 None None] MFA Rules not processed for user `6362303cb918486c8525225914332ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:14.766100 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-12d1e128-ae24-48e8-8c40-3e68f3c5eae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:14.779838 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f7d76-60dd-45d7-ba5e-70e6fe5c207d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:14.782618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f7d76-60dd-45d7-ba5e-70e6fe5c207d admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:14.783810 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f7d76-60dd-45d7-ba5e-70e6fe5c207d admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:14.784092 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f7d76-60dd-45d7-ba5e-70e6fe5c207d admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:14.792024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f7d76-60dd-45d7-ba5e-70e6fe5c207d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:14.797554 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f7d76-60dd-45d7-ba5e-70e6fe5c207d admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:14.857686 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 821/1625] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:14.858977 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f7d76-60dd-45d7-ba5e-70e6fe5c207d admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:14.897506 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f7d76-60dd-45d7-ba5e-70e6fe5c207d admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:14.914515 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b8721-97bb-4be7-80c2-dcd9297e5994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:14.916820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b8721-97bb-4be7-80c2-dcd9297e5994 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:14.917137 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b8721-97bb-4be7-80c2-dcd9297e5994 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:14.917410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b8721-97bb-4be7-80c2-dcd9297e5994 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:14.919947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b8721-97bb-4be7-80c2-dcd9297e5994 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:14.923245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 805/1626] 10.222.0.26 () {58 vars in 1058 bytes} [Mon May 9 09:47:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:14.925346 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1e128-ae24-48e8-8c40-3e68f3c5eae8 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:14.926646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b8721-97bb-4be7-80c2-dcd9297e5994 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:14.952975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1e128-ae24-48e8-8c40-3e68f3c5eae8 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:14.960252 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a83363-1117-4e29-bf56-45cd6d5422eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:14.961409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a83363-1117-4e29-bf56-45cd6d5422eb admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:14.961686 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a83363-1117-4e29-bf56-45cd6d5422eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:14.961991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a83363-1117-4e29-bf56-45cd6d5422eb admin admin] PATH_INFO: `/v3/projects/153df24b6b9c4bc7937e0e94bdc8fdf0/users/b55f70c0a7964afea131d9ec1d984fc2/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:14.975348 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 822/1627] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:14 2022] POST /identity/v3/users => generated 308 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:14.977515 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b8721-97bb-4be7-80c2-dcd9297e5994 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:15.000735 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b8721-97bb-4be7-80c2-dcd9297e5994 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:15.009651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a83363-1117-4e29-bf56-45cd6d5422eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=153df24b6b9c4bc7937e0e94bdc8fdf0, user_id=b55f70c0a7964afea131d9ec1d984fc2, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:15.014444 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b908704-3aa9-41c6-bd15-85f6be38b95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:15.016818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b908704-3aa9-41c6-bd15-85f6be38b95c admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:15.017102 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b908704-3aa9-41c6-bd15-85f6be38b95c admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:15.017307 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a83363-1117-4e29-bf56-45cd6d5422eb admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:15.018068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b908704-3aa9-41c6-bd15-85f6be38b95c admin admin] PATH_INFO: `/v3/projects/b5b4118142524da281c8ba137e833456/users/843391e07bc0400eb41661ea0b2b1963/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:15.022966 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b908704-3aa9-41c6-bd15-85f6be38b95c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5b4118142524da281c8ba137e833456, user_id=843391e07bc0400eb41661ea0b2b1963, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:15.027520 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b908704-3aa9-41c6-bd15-85f6be38b95c admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:15.044509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 806/1628] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:14 2022] PUT /identity/v3/projects/153df24b6b9c4bc7937e0e94bdc8fdf0/users/b55f70c0a7964afea131d9ec1d984fc2/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:15.046711 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a83363-1117-4e29-bf56-45cd6d5422eb admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:15.073019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 823/1629] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:14 2022] PUT /identity/v3/projects/b5b4118142524da281c8ba137e833456/users/843391e07bc0400eb41661ea0b2b1963/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:15.075241 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b908704-3aa9-41c6-bd15-85f6be38b95c admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:15.226104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b908704-3aa9-41c6-bd15-85f6be38b95c admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:15.233964 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a83363-1117-4e29-bf56-45cd6d5422eb admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:15.237858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e7108-2c69-44dd-8e76-26d0aa5a5e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:15.240999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e7108-2c69-44dd-8e76-26d0aa5a5e99 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:15.241281 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e7108-2c69-44dd-8e76-26d0aa5a5e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:15.241556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e7108-2c69-44dd-8e76-26d0aa5a5e99 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:15.244177 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e7108-2c69-44dd-8e76-26d0aa5a5e99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:15.247092 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e7108-2c69-44dd-8e76-26d0aa5a5e99 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:15.286953 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 824/1630] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:15.289566 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e7108-2c69-44dd-8e76-26d0aa5a5e99 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:15.314300 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e7108-2c69-44dd-8e76-26d0aa5a5e99 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:15.325790 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919e9b0-9ae0-45bc-b510-155b23fc96d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:15.326936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919e9b0-9ae0-45bc-b510-155b23fc96d5 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:15.328026 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919e9b0-9ae0-45bc-b510-155b23fc96d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:15.328354 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919e9b0-9ae0-45bc-b510-155b23fc96d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:15.331205 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1919e9b0-9ae0-45bc-b510-155b23fc96d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:15.332894 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1919e9b0-9ae0-45bc-b510-155b23fc96d5 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:15.356314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 825/1631] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:15.357913 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08132a0d-45ad-4f31-ab78-835b14fe931d None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:15.358232 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08132a0d-45ad-4f31-ab78-835b14fe931d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:15.358681 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08132a0d-45ad-4f31-ab78-835b14fe931d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:15.405318 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39974c-d244-46ce-859c-79095d5d9b03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:15.406042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39974c-d244-46ce-859c-79095d5d9b03 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:15.406321 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39974c-d244-46ce-859c-79095d5d9b03 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:15.406720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39974c-d244-46ce-859c-79095d5d9b03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:15.430393 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-08132a0d-45ad-4f31-ab78-835b14fe931d None None] MFA Rules not processed for user `843391e07bc0400eb41661ea0b2b1963`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:15.431978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-08132a0d-45ad-4f31-ab78-835b14fe931d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:15.530333 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39974c-d244-46ce-859c-79095d5d9b03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:15.534190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39974c-d244-46ce-859c-79095d5d9b03 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:15.560433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 807/1632] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:15.561350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39974c-d244-46ce-859c-79095d5d9b03 service nova] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:15.584905 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39974c-d244-46ce-859c-79095d5d9b03 service nova] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:15.596425 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f259c54-9d87-42e5-890c-640af91a7cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:15.598493 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f259c54-9d87-42e5-890c-640af91a7cfd admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:15.598771 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f259c54-9d87-42e5-890c-640af91a7cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:15.611402 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 826/1633] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:15.612274 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f259c54-9d87-42e5-890c-640af91a7cfd admin admin] PATH_INFO: `/v3/projects/67065741880948658178d1136b4d9d67/users/6faaad80998e4fae92f0e8b75f7ceba7/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:15.613483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08132a0d-45ad-4f31-ab78-835b14fe931d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:15.638811 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f259c54-9d87-42e5-890c-640af91a7cfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=67065741880948658178d1136b4d9d67, user_id=6faaad80998e4fae92f0e8b75f7ceba7, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:15.641165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08132a0d-45ad-4f31-ab78-835b14fe931d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:15.641654 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f259c54-9d87-42e5-890c-640af91a7cfd admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:15.657464 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db58ecf-b5c0-49de-ac18-3bdbf828f3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:15.658679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db58ecf-b5c0-49de-ac18-3bdbf828f3f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:15.659142 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db58ecf-b5c0-49de-ac18-3bdbf828f3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:15.662104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 808/1634] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:15 2022] PUT /identity/v3/projects/67065741880948658178d1136b4d9d67/users/6faaad80998e4fae92f0e8b75f7ceba7/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:15.662104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f259c54-9d87-42e5-890c-640af91a7cfd admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:15.662886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db58ecf-b5c0-49de-ac18-3bdbf828f3f1 admin admin] PATH_INFO: `/v3/projects/153df24b6b9c4bc7937e0e94bdc8fdf0/users/b55f70c0a7964afea131d9ec1d984fc2/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:15.668272 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db58ecf-b5c0-49de-ac18-3bdbf828f3f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=153df24b6b9c4bc7937e0e94bdc8fdf0, user_id=b55f70c0a7964afea131d9ec1d984fc2, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:15.671202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db58ecf-b5c0-49de-ac18-3bdbf828f3f1 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:15.702885 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 827/1635] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:15 2022] PUT /identity/v3/projects/153df24b6b9c4bc7937e0e94bdc8fdf0/users/b55f70c0a7964afea131d9ec1d984fc2/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:15.704332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db58ecf-b5c0-49de-ac18-3bdbf828f3f1 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:15.734972 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f259c54-9d87-42e5-890c-640af91a7cfd admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:15.800648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ce9d7-4a36-40c8-8ee3-fad0988fcddb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:15.801838 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ce9d7-4a36-40c8-8ee3-fad0988fcddb service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:15.802115 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ce9d7-4a36-40c8-8ee3-fad0988fcddb service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:15.802767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ce9d7-4a36-40c8-8ee3-fad0988fcddb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:15.859011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db58ecf-b5c0-49de-ac18-3bdbf828f3f1 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:15.864891 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabdee3-3ef5-42d2-8db3-b10837c30122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:15.865834 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabdee3-3ef5-42d2-8db3-b10837c30122 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:15.866884 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabdee3-3ef5-42d2-8db3-b10837c30122 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:15.867384 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabdee3-3ef5-42d2-8db3-b10837c30122 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:15.868496 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aabdee3-3ef5-42d2-8db3-b10837c30122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:15.870906 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aabdee3-3ef5-42d2-8db3-b10837c30122 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:15.872833 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ce9d7-4a36-40c8-8ee3-fad0988fcddb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:15.874421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ce9d7-4a36-40c8-8ee3-fad0988fcddb service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:15.894907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 828/1636] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:15.896512 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556053b9-0a70-4b54-8951-37b9653f1fe9 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:15.897588 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 809/1637] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:15.899457 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556053b9-0a70-4b54-8951-37b9653f1fe9 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:15.899973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556053b9-0a70-4b54-8951-37b9653f1fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:15.900105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ce9d7-4a36-40c8-8ee3-fad0988fcddb service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:15.919024 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ce9d7-4a36-40c8-8ee3-fad0988fcddb service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:15.944038 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec613451-3532-4a2e-a017-acb2a4805059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:15.944988 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec613451-3532-4a2e-a017-acb2a4805059 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:15.945266 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec613451-3532-4a2e-a017-acb2a4805059 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:15.945533 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec613451-3532-4a2e-a017-acb2a4805059 admin admin] PATH_INFO: `/v3/projects/67065741880948658178d1136b4d9d67/users/6faaad80998e4fae92f0e8b75f7ceba7/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:15.947523 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec613451-3532-4a2e-a017-acb2a4805059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67065741880948658178d1136b4d9d67, user_id=6faaad80998e4fae92f0e8b75f7ceba7, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:15.950988 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec613451-3532-4a2e-a017-acb2a4805059 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:15.961855 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-556053b9-0a70-4b54-8951-37b9653f1fe9 None None] MFA Rules not processed for user `b55f70c0a7964afea131d9ec1d984fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:15.963968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 810/1638] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:15 2022] PUT /identity/v3/projects/67065741880948658178d1136b4d9d67/users/6faaad80998e4fae92f0e8b75f7ceba7/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:15.965504 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-556053b9-0a70-4b54-8951-37b9653f1fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:15.976123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4e181-aa8d-4ca7-90c0-36cacb686366 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:15.976712 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4e181-aa8d-4ca7-90c0-36cacb686366 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:15.977093 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4e181-aa8d-4ca7-90c0-36cacb686366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:16.039820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-04b4e181-aa8d-4ca7-90c0-36cacb686366 None None] MFA Rules not processed for user `6faaad80998e4fae92f0e8b75f7ceba7`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:16.041324 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-04b4e181-aa8d-4ca7-90c0-36cacb686366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:16.128174 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 829/1639] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:16.130145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556053b9-0a70-4b54-8951-37b9653f1fe9 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:16.184982 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 811/1640] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:15 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:16.185915 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4e181-aa8d-4ca7-90c0-36cacb686366 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:16.195573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556053b9-0a70-4b54-8951-37b9653f1fe9 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:16.201361 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d3109-9bb1-440d-adf3-407df3c48d2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:16.202742 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d3109-9bb1-440d-adf3-407df3c48d2a service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:16.203252 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d3109-9bb1-440d-adf3-407df3c48d2a service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:16.203800 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d3109-9bb1-440d-adf3-407df3c48d2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:16.209145 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4e181-aa8d-4ca7-90c0-36cacb686366 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:16.214194 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4daffb-5625-4483-a3a3-7588a4e5cab2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:16.215284 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4daffb-5625-4483-a3a3-7588a4e5cab2 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:16.215887 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4daffb-5625-4483-a3a3-7588a4e5cab2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:16.216761 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4daffb-5625-4483-a3a3-7588a4e5cab2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:16.305228 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4daffb-5625-4483-a3a3-7588a4e5cab2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:16.311634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4daffb-5625-4483-a3a3-7588a4e5cab2 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:16.314442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d3109-9bb1-440d-adf3-407df3c48d2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:16.316057 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d3109-9bb1-440d-adf3-407df3c48d2a service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:16.327795 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 812/1641] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:16.329488 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950742b-7cc0-48e3-9555-8feefb15138b None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:16.330003 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950742b-7cc0-48e3-9555-8feefb15138b None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:16.330514 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950742b-7cc0-48e3-9555-8feefb15138b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:16.351109 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 830/1642] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:16.379382 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-2950742b-7cc0-48e3-9555-8feefb15138b None None] MFA Rules not processed for user `40cb55020f07488c9c344541d895ec1c`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:16.386373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-2950742b-7cc0-48e3-9555-8feefb15138b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:16.647550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 813/1643] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:16.660389 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8818c-ebd3-44aa-83f3-b1061e48bdf3 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:16.660778 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8818c-ebd3-44aa-83f3-b1061e48bdf3 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:16.662693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8818c-ebd3-44aa-83f3-b1061e48bdf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:16.688743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f324c1a-c41a-4689-8bbb-a4cb71071075 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:16.689257 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f324c1a-c41a-4689-8bbb-a4cb71071075 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:16.689662 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f324c1a-c41a-4689-8bbb-a4cb71071075 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:16.691397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 814/1644] 10.222.0.26 () {54 vars in 961 bytes} [Mon May 9 09:47:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:47:16.705246 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23851bcf-3f7e-406a-91e6-0c602a9efcc8 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:16.706260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23851bcf-3f7e-406a-91e6-0c602a9efcc8 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:16.706562 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23851bcf-3f7e-406a-91e6-0c602a9efcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:16.749780 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-28f8818c-ebd3-44aa-83f3-b1061e48bdf3 None None] MFA Rules not processed for user `40cb55020f07488c9c344541d895ec1c`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:16.754678 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-28f8818c-ebd3-44aa-83f3-b1061e48bdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:16.764815 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-23851bcf-3f7e-406a-91e6-0c602a9efcc8 None None] MFA Rules not processed for user `6362303cb918486c8525225914332ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:16.766188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-23851bcf-3f7e-406a-91e6-0c602a9efcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:16.780286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 831/1645] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:16.794385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8818c-ebd3-44aa-83f3-b1061e48bdf3 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:16.917728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 815/1646] 10.222.0.26 () {58 vars in 1058 bytes} [Mon May 9 09:47:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:16.920416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83221381-dde1-4811-8b27-0c5e3e33be29 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:16.920734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83221381-dde1-4811-8b27-0c5e3e33be29 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:16.921014 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83221381-dde1-4811-8b27-0c5e3e33be29 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:16.941057 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 816/1647] 10.222.0.26 () {54 vars in 912 bytes} [Mon May 9 09:47:16 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:47:16.946796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6891d-9337-4838-b122-1ac0469cffdb None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:16.947084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6891d-9337-4838-b122-1ac0469cffdb None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:16.947365 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6891d-9337-4838-b122-1ac0469cffdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:17.000654 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8818c-ebd3-44aa-83f3-b1061e48bdf3 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:17.014419 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-74d6891d-9337-4838-b122-1ac0469cffdb None None] MFA Rules not processed for user `6362303cb918486c8525225914332ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:17.017662 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1480e-3f74-4d3a-bf23-b1d440b2c220 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:17.019829 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1480e-3f74-4d3a-bf23-b1d440b2c220 service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:17.022620 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1480e-3f74-4d3a-bf23-b1d440b2c220 service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:17.022620 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1480e-3f74-4d3a-bf23-b1d440b2c220 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:17.023042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-74d6891d-9337-4838-b122-1ac0469cffdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:17.048542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b1480e-3f74-4d3a-bf23-b1d440b2c220 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:17.052872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b1480e-3f74-4d3a-bf23-b1d440b2c220 service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:17.056373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 817/1648] 10.222.0.26 () {58 vars in 1009 bytes} [Mon May 9 09:47:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:17.058193 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6891d-9337-4838-b122-1ac0469cffdb None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:17.078519 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 832/1649] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 09:47:17.081513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6891d-9337-4838-b122-1ac0469cffdb None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:17.096039 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb546f51-0159-4456-93df-f8bc54cc5741 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:17.097114 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb546f51-0159-4456-93df-f8bc54cc5741 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:17.097114 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb546f51-0159-4456-93df-f8bc54cc5741 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:17.097318 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb546f51-0159-4456-93df-f8bc54cc5741 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:17.113521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb546f51-0159-4456-93df-f8bc54cc5741 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:17.117849 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb546f51-0159-4456-93df-f8bc54cc5741 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:17.149622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 818/1650] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:17 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:17.211987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee631bfd-edd0-4c6e-a735-bc94c1d01cba None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:17.212484 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee631bfd-edd0-4c6e-a735-bc94c1d01cba None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:17.212917 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee631bfd-edd0-4c6e-a735-bc94c1d01cba None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:17.218398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 833/1651] 10.222.0.26 () {54 vars in 912 bytes} [Mon May 9 09:47:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:47:17.231758 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5111ebb-7693-4946-a388-71448e4577b7 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:17.233469 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5111ebb-7693-4946-a388-71448e4577b7 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:17.233844 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5111ebb-7693-4946-a388-71448e4577b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:17.266788 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-a5111ebb-7693-4946-a388-71448e4577b7 None None] MFA Rules not processed for user `6362303cb918486c8525225914332ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:17.269222 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a5111ebb-7693-4946-a388-71448e4577b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:17.289299 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 819/1652] 10.222.0.26 () {58 vars in 1009 bytes} [Mon May 9 09:47:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:17.304333 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee631bfd-edd0-4c6e-a735-bc94c1d01cba None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:17.306350 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-ee631bfd-edd0-4c6e-a735-bc94c1d01cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:17.477773 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee631bfd-edd0-4c6e-a735-bc94c1d01cba None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:17.487004 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f227b-68f0-4c8f-b8ef-8bb04c78a383 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fd398c547554629927d2a80b3a8dee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:17.487859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f227b-68f0-4c8f-b8ef-8bb04c78a383 service glance] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:17.489855 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f227b-68f0-4c8f-b8ef-8bb04c78a383 service glance] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:17.490253 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f227b-68f0-4c8f-b8ef-8bb04c78a383 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:17.525015 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f227b-68f0-4c8f-b8ef-8bb04c78a383 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:17.534247 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f227b-68f0-4c8f-b8ef-8bb04c78a383 service glance] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:17.558782 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 834/1653] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:17.665623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5111ebb-7693-4946-a388-71448e4577b7 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:17.677132 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5111ebb-7693-4946-a388-71448e4577b7 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:17.693673 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54273f2a-22c4-45f0-a36d-eacf38bb6500 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:17.695351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54273f2a-22c4-45f0-a36d-eacf38bb6500 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:17.695980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54273f2a-22c4-45f0-a36d-eacf38bb6500 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:17.696436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54273f2a-22c4-45f0-a36d-eacf38bb6500 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:17.711636 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54273f2a-22c4-45f0-a36d-eacf38bb6500 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:17.713812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54273f2a-22c4-45f0-a36d-eacf38bb6500 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:17.789230 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 820/1654] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:18.267179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f77bcd-a011-4e69-942b-bdf39b51b827 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:18.268166 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f77bcd-a011-4e69-942b-bdf39b51b827 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:18.268648 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f77bcd-a011-4e69-942b-bdf39b51b827 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:18.270646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 835/1655] 10.222.0.26 () {54 vars in 912 bytes} [Mon May 9 09:47:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:47:18.276436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9b8b0-1c46-421f-aa47-8c0a77d5e502 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:18.277080 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9b8b0-1c46-421f-aa47-8c0a77d5e502 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:18.277080 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9b8b0-1c46-421f-aa47-8c0a77d5e502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:18.325763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f77bcd-a011-4e69-942b-bdf39b51b827 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:18.373425 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-36e9b8b0-1c46-421f-aa47-8c0a77d5e502 None None] MFA Rules not processed for user `0f0d3f4264a04093894d6bc5c5c45c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:18.380535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-36e9b8b0-1c46-421f-aa47-8c0a77d5e502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:18.447909 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f77bcd-a011-4e69-942b-bdf39b51b827 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:18.453405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715f082-4201-435a-a0cb-236e442b4a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:18.454579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715f082-4201-435a-a0cb-236e442b4a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:18.454859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715f082-4201-435a-a0cb-236e442b4a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:18.455133 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715f082-4201-435a-a0cb-236e442b4a3b admin admin] PATH_INFO: `/v3/users/31fb080fb94046f8bc64575c367f5ec8` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:18.457951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d715f082-4201-435a-a0cb-236e442b4a3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=31fb080fb94046f8bc64575c367f5ec8)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:18.461735 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d715f082-4201-435a-a0cb-236e442b4a3b admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:18.479197 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 821/1656] 10.222.0.26 () {58 vars in 1009 bytes} [Mon May 9 09:47:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:18.481891 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9b8b0-1c46-421f-aa47-8c0a77d5e502 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:18.507315 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9b8b0-1c46-421f-aa47-8c0a77d5e502 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:18.517264 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb7e8e-f4a1-4a3c-ad25-fd0ad42edbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:18.518727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb7e8e-f4a1-4a3c-ad25-fd0ad42edbd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:18.519008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb7e8e-f4a1-4a3c-ad25-fd0ad42edbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:18.519629 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb7e8e-f4a1-4a3c-ad25-fd0ad42edbd4 admin admin] PATH_INFO: `/v3/users/8069b466c6354d6d96bfd1bdb5f7710b` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:18.522793 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdb7e8e-f4a1-4a3c-ad25-fd0ad42edbd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8069b466c6354d6d96bfd1bdb5f7710b)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:18.524228 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: WARNING py.warnings [None req-afdb7e8e-f4a1-4a3c-ad25-fd0ad42edbd4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 09:47:18.524228 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: warnings.warn(msg) May 09 09:47:18.524228 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: May 09 09:47:18.526692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdb7e8e-f4a1-4a3c-ad25-fd0ad42edbd4 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:18.578110 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-d715f082-4201-435a-a0cb-236e442b4a3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31fb080fb94046f8bc64575c367f5ec8'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:18.587785 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-d715f082-4201-435a-a0cb-236e442b4a3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31fb080fb94046f8bc64575c367f5ec8'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:18.588904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-d715f082-4201-435a-a0cb-236e442b4a3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31fb080fb94046f8bc64575c367f5ec8'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:18.607219 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-d715f082-4201-435a-a0cb-236e442b4a3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31fb080fb94046f8bc64575c367f5ec8'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:18.613447 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-d715f082-4201-435a-a0cb-236e442b4a3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31fb080fb94046f8bc64575c367f5ec8'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:18.642019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 836/1657] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:47:18 2022] DELETE /identity/v3/users/31fb080fb94046f8bc64575c367f5ec8 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:47:18.643198 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715f082-4201-435a-a0cb-236e442b4a3b admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:18.655439 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-afdb7e8e-f4a1-4a3c-ad25-fd0ad42edbd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8069b466c6354d6d96bfd1bdb5f7710b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:18.667222 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-d715f082-4201-435a-a0cb-236e442b4a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:18.670862 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-afdb7e8e-f4a1-4a3c-ad25-fd0ad42edbd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8069b466c6354d6d96bfd1bdb5f7710b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:18.690759 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-afdb7e8e-f4a1-4a3c-ad25-fd0ad42edbd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8069b466c6354d6d96bfd1bdb5f7710b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:18.692229 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-afdb7e8e-f4a1-4a3c-ad25-fd0ad42edbd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8069b466c6354d6d96bfd1bdb5f7710b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:18.693135 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-afdb7e8e-f4a1-4a3c-ad25-fd0ad42edbd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8069b466c6354d6d96bfd1bdb5f7710b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:18.770843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 822/1658] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:47:18 2022] DELETE /identity/v3/users/8069b466c6354d6d96bfd1bdb5f7710b => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:18.771506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25206c74-c67b-4806-969a-bc8983770c51 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:18.773458 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25206c74-c67b-4806-969a-bc8983770c51 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:18.773808 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25206c74-c67b-4806-969a-bc8983770c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:18.814065 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715f082-4201-435a-a0cb-236e442b4a3b admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:18.824384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-25206c74-c67b-4806-969a-bc8983770c51 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:18.829105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-25206c74-c67b-4806-969a-bc8983770c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:18.868172 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be72310-c40a-4509-a1df-c6fbcfd323f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:18.869026 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be72310-c40a-4509-a1df-c6fbcfd323f4 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:18.869378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be72310-c40a-4509-a1df-c6fbcfd323f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:18.869726 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be72310-c40a-4509-a1df-c6fbcfd323f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:18.871952 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-0be72310-c40a-4509-a1df-c6fbcfd323f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:18.882832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be72310-c40a-4509-a1df-c6fbcfd323f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:18.884676 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be72310-c40a-4509-a1df-c6fbcfd323f4 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:18.893890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 837/1659] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:18.895568 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb306fa-846d-44ac-a5f9-7b023043c2fd None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:18.895917 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb306fa-846d-44ac-a5f9-7b023043c2fd None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:18.896265 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb306fa-846d-44ac-a5f9-7b023043c2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:18.916224 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-4fb306fa-846d-44ac-a5f9-7b023043c2fd None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:18.917583 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-4fb306fa-846d-44ac-a5f9-7b023043c2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:18.988500 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 823/1660] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:19.020363 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 838/1661] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:19.023290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25206c74-c67b-4806-969a-bc8983770c51 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:19.026755 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-25206c74-c67b-4806-969a-bc8983770c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:19.033228 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb306fa-846d-44ac-a5f9-7b023043c2fd None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:19.035577 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-4fb306fa-846d-44ac-a5f9-7b023043c2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:19.048213 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25206c74-c67b-4806-969a-bc8983770c51 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:19.067179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb306fa-846d-44ac-a5f9-7b023043c2fd None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:19.072172 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61aae53-3c99-4744-bdb8-d5282085617d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:19.073695 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61aae53-3c99-4744-bdb8-d5282085617d service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:19.075090 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61aae53-3c99-4744-bdb8-d5282085617d service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:19.076607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61aae53-3c99-4744-bdb8-d5282085617d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:19.090309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247b6b5-627d-4a9c-b08d-fc0410aba7ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:19.091000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247b6b5-627d-4a9c-b08d-fc0410aba7ad service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:19.091282 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247b6b5-627d-4a9c-b08d-fc0410aba7ad service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:19.091557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247b6b5-627d-4a9c-b08d-fc0410aba7ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:19.094757 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61aae53-3c99-4744-bdb8-d5282085617d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:19.097275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61aae53-3c99-4744-bdb8-d5282085617d service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:19.107779 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8247b6b5-627d-4a9c-b08d-fc0410aba7ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:19.109312 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8247b6b5-627d-4a9c-b08d-fc0410aba7ad service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:19.120918 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 824/1662] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:19.130372 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 839/1663] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:20.455684 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d5100-b983-4b19-be20-33707e1bb8b6 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:20.456371 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d5100-b983-4b19-be20-33707e1bb8b6 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:20.456371 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d5100-b983-4b19-be20-33707e1bb8b6 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:20.457954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 825/1664] 10.222.0.26 () {54 vars in 960 bytes} [Mon May 9 09:47:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:47:20.465418 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405efed-aeb8-4ec6-bc54-f0b7ce134b40 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:20.466254 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405efed-aeb8-4ec6-bc54-f0b7ce134b40 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:20.466708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405efed-aeb8-4ec6-bc54-f0b7ce134b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:20.499068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-7405efed-aeb8-4ec6-bc54-f0b7ce134b40 None None] MFA Rules not processed for user `6c9e458c2c5f4ed1b3eddd84c2ec1703`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:20.501440 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-7405efed-aeb8-4ec6-bc54-f0b7ce134b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:20.609827 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 840/1665] 10.222.0.26 () {58 vars in 1057 bytes} [Mon May 9 09:47:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:20.618102 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d5100-b983-4b19-be20-33707e1bb8b6 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:20.638508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d5100-b983-4b19-be20-33707e1bb8b6 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:20.651281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b16afc0-7543-43ee-93c4-9cd2c943b432 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9e458c2c5f4ed1b3eddd84c2ec1703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:20.653505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16afc0-7543-43ee-93c4-9cd2c943b432 service placement] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:20.654800 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16afc0-7543-43ee-93c4-9cd2c943b432 service placement] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:20.656232 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16afc0-7543-43ee-93c4-9cd2c943b432 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:20.666609 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-5b16afc0-7543-43ee-93c4-9cd2c943b432 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:20.698736 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b16afc0-7543-43ee-93c4-9cd2c943b432 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:20.700856 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b16afc0-7543-43ee-93c4-9cd2c943b432 service placement] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:20.715817 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 826/1666] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:21.034959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405efed-aeb8-4ec6-bc54-f0b7ce134b40 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:21.036434 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-7405efed-aeb8-4ec6-bc54-f0b7ce134b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:21.068650 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405efed-aeb8-4ec6-bc54-f0b7ce134b40 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:21.083026 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d2387-ba88-429a-a283-3596b45b81a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:21.083929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d2387-ba88-429a-a283-3596b45b81a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:21.084202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d2387-ba88-429a-a283-3596b45b81a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:21.084471 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d2387-ba88-429a-a283-3596b45b81a8 admin admin] PATH_INFO: `/v3/projects/ccaa0efe9f4d44d8899fabe0d5ed3d86` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:21.088342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d2387-ba88-429a-a283-3596b45b81a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccaa0efe9f4d44d8899fabe0d5ed3d86)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:21.091089 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: WARNING py.warnings [None req-887d2387-ba88-429a-a283-3596b45b81a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 09:47:21.091089 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: warnings.warn(msg) May 09 09:47:21.091089 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: May 09 09:47:21.092295 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d2387-ba88-429a-a283-3596b45b81a8 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:21.275399 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-887d2387-ba88-429a-a283-3596b45b81a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccaa0efe9f4d44d8899fabe0d5ed3d86', 'request_id': 'req-887d2387-ba88-429a-a283-3596b45b81a8'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:21.304930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-887d2387-ba88-429a-a283-3596b45b81a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccaa0efe9f4d44d8899fabe0d5ed3d86', 'request_id': 'req-887d2387-ba88-429a-a283-3596b45b81a8'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:21.320823 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-887d2387-ba88-429a-a283-3596b45b81a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccaa0efe9f4d44d8899fabe0d5ed3d86', 'request_id': 'req-887d2387-ba88-429a-a283-3596b45b81a8'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:21.352657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-887d2387-ba88-429a-a283-3596b45b81a8 admin admin] The token cache is being invalidate because project ccaa0efe9f4d44d8899fabe0d5ed3d86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:47:21.352973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-887d2387-ba88-429a-a283-3596b45b81a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:21.353795 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-887d2387-ba88-429a-a283-3596b45b81a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:21.359779 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 841/1667] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:47:21 2022] DELETE /identity/v3/projects/ccaa0efe9f4d44d8899fabe0d5ed3d86 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:21.364833 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def74c9-657c-4ac9-ac5d-965d02573b11 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:21.365122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def74c9-657c-4ac9-ac5d-965d02573b11 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:21.365442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def74c9-657c-4ac9-ac5d-965d02573b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:21.398113 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-1def74c9-657c-4ac9-ac5d-965d02573b11 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:21.399376 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-1def74c9-657c-4ac9-ac5d-965d02573b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:21.490087 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 827/1668] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:21.491754 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db60cf-3d67-441b-a94c-6683f28ff538 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:21.492195 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db60cf-3d67-441b-a94c-6683f28ff538 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:21.492297 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db60cf-3d67-441b-a94c-6683f28ff538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:21.524854 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-b6db60cf-3d67-441b-a94c-6683f28ff538 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:21.526810 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b6db60cf-3d67-441b-a94c-6683f28ff538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:21.538201 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 842/1669] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:21.543252 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def74c9-657c-4ac9-ac5d-965d02573b11 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:21.572200 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def74c9-657c-4ac9-ac5d-965d02573b11 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:21.580316 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f92ee-5222-44c1-827d-c38744769cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:21.581905 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f92ee-5222-44c1-827d-c38744769cac admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:21.582288 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f92ee-5222-44c1-827d-c38744769cac admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:21.582603 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f92ee-5222-44c1-827d-c38744769cac admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:21.583492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f92ee-5222-44c1-827d-c38744769cac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:47:21.584463 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f92ee-5222-44c1-827d-c38744769cac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:21.586836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f92ee-5222-44c1-827d-c38744769cac admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:21.602553 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 828/1670] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:47:21.606700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db60cf-3d67-441b-a94c-6683f28ff538 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:21.619731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db60cf-3d67-441b-a94c-6683f28ff538 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:21.638365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcef36b-e805-4f3a-9dd2-4607a906c5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:21.639864 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcef36b-e805-4f3a-9dd2-4607a906c5b4 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:21.640277 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcef36b-e805-4f3a-9dd2-4607a906c5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:21.641429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcef36b-e805-4f3a-9dd2-4607a906c5b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:21.642348 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcef36b-e805-4f3a-9dd2-4607a906c5b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:21.647393 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcef36b-e805-4f3a-9dd2-4607a906c5b4 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:21.692764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 843/1671] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:21 2022] POST /identity/v3/projects => generated 393 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:21.698731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f92ee-5222-44c1-827d-c38744769cac admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:21.715690 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f92ee-5222-44c1-827d-c38744769cac admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:21.845815 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb11c3-d911-436d-acc4-f7e849d75320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:21.846508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb11c3-d911-436d-acc4-f7e849d75320 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:21.846787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb11c3-d911-436d-acc4-f7e849d75320 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:21.847056 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb11c3-d911-436d-acc4-f7e849d75320 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:21.847757 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb11c3-d911-436d-acc4-f7e849d75320 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:21.849945 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb11c3-d911-436d-acc4-f7e849d75320 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:21.895847 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 829/1672] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:21 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:21.899723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcef36b-e805-4f3a-9dd2-4607a906c5b4 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:21.919826 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcef36b-e805-4f3a-9dd2-4607a906c5b4 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:21.935187 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab4f55-b96c-4485-a702-f6fdeeffd8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:21.939585 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab4f55-b96c-4485-a702-f6fdeeffd8b1 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:21.939842 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab4f55-b96c-4485-a702-f6fdeeffd8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:21.940120 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab4f55-b96c-4485-a702-f6fdeeffd8b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:21.943346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab4f55-b96c-4485-a702-f6fdeeffd8b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:21.945082 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab4f55-b96c-4485-a702-f6fdeeffd8b1 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:21.974848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 844/1673] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:47:21.978943 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb11c3-d911-436d-acc4-f7e849d75320 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:21.990494 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb11c3-d911-436d-acc4-f7e849d75320 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:22.022430 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310e7f3-69c2-4528-8c54-b6c4d73de832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:22.022806 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310e7f3-69c2-4528-8c54-b6c4d73de832 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:22.023021 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310e7f3-69c2-4528-8c54-b6c4d73de832 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:22.023299 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310e7f3-69c2-4528-8c54-b6c4d73de832 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:22.024155 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310e7f3-69c2-4528-8c54-b6c4d73de832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:22.025849 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310e7f3-69c2-4528-8c54-b6c4d73de832 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:22.062108 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 830/1674] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:22.064617 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab4f55-b96c-4485-a702-f6fdeeffd8b1 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:22.083119 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab4f55-b96c-4485-a702-f6fdeeffd8b1 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:22.088555 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310e7f3-69c2-4528-8c54-b6c4d73de832 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:22.090338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22047d-9b74-4b63-b099-5646eca1b055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:22.092075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22047d-9b74-4b63-b099-5646eca1b055 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:22.092498 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22047d-9b74-4b63-b099-5646eca1b055 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:22.093436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e310e7f3-69c2-4528-8c54-b6c4d73de832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:22.093990 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22047d-9b74-4b63-b099-5646eca1b055 admin admin] PATH_INFO: `/v3/projects/005420558a4a41ec904e88728581fd5e/users/9f2d5d83d69f4e6f8df4a8192aa30880/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:22.121267 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b22047d-9b74-4b63-b099-5646eca1b055 admin admin] RBAC: Authorizing `identity:create_grant(project_id=005420558a4a41ec904e88728581fd5e, user_id=9f2d5d83d69f4e6f8df4a8192aa30880, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:22.125968 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b22047d-9b74-4b63-b099-5646eca1b055 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:22.130148 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310e7f3-69c2-4528-8c54-b6c4d73de832 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:22.151306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad0660-e022-40af-a34b-825e51f360f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:22.152430 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad0660-e022-40af-a34b-825e51f360f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:22.152909 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad0660-e022-40af-a34b-825e51f360f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:22.153332 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad0660-e022-40af-a34b-825e51f360f5 admin admin] PATH_INFO: `/v3/projects/e9aaa358e62342b1a17858f9c1356656` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:22.153562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 845/1675] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:22 2022] PUT /identity/v3/projects/005420558a4a41ec904e88728581fd5e/users/9f2d5d83d69f4e6f8df4a8192aa30880/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:22.157508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad0660-e022-40af-a34b-825e51f360f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9aaa358e62342b1a17858f9c1356656)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:22.159797 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad0660-e022-40af-a34b-825e51f360f5 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:22.165191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1badea7-cb46-4bfd-821f-50f30ebbfc22 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:22.165476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1badea7-cb46-4bfd-821f-50f30ebbfc22 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:22.165752 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1badea7-cb46-4bfd-821f-50f30ebbfc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:22.231155 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-e1badea7-cb46-4bfd-821f-50f30ebbfc22 None None] MFA Rules not processed for user `9f2d5d83d69f4e6f8df4a8192aa30880`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:22.235380 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e1badea7-cb46-4bfd-821f-50f30ebbfc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:22.353279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-57ad0660-e022-40af-a34b-825e51f360f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9aaa358e62342b1a17858f9c1356656', 'request_id': 'req-57ad0660-e022-40af-a34b-825e51f360f5'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:22.370810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-57ad0660-e022-40af-a34b-825e51f360f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9aaa358e62342b1a17858f9c1356656', 'request_id': 'req-57ad0660-e022-40af-a34b-825e51f360f5'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:22.377853 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-57ad0660-e022-40af-a34b-825e51f360f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9aaa358e62342b1a17858f9c1356656', 'request_id': 'req-57ad0660-e022-40af-a34b-825e51f360f5'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:22.415336 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 846/1676] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:22.418677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1badea7-cb46-4bfd-821f-50f30ebbfc22 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:22.427461 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-57ad0660-e022-40af-a34b-825e51f360f5 admin admin] The token cache is being invalidate because project e9aaa358e62342b1a17858f9c1356656 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:47:22.427762 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-57ad0660-e022-40af-a34b-825e51f360f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:22.433994 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-57ad0660-e022-40af-a34b-825e51f360f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:22.459581 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 831/1677] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:47:22 2022] DELETE /identity/v3/projects/e9aaa358e62342b1a17858f9c1356656 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:22.460610 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad0660-e022-40af-a34b-825e51f360f5 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:22.463760 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-57ad0660-e022-40af-a34b-825e51f360f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:22.518615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1badea7-cb46-4bfd-821f-50f30ebbfc22 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:22.634543 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad0660-e022-40af-a34b-825e51f360f5 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:22.645434 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bcecfb-3883-4a57-9a70-3cc4a5273141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:22.646578 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bcecfb-3883-4a57-9a70-3cc4a5273141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:22.647168 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bcecfb-3883-4a57-9a70-3cc4a5273141 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:22.647595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bcecfb-3883-4a57-9a70-3cc4a5273141 admin admin] PATH_INFO: `/v3/users/6faaad80998e4fae92f0e8b75f7ceba7` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:22.649338 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bcecfb-3883-4a57-9a70-3cc4a5273141 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6faaad80998e4fae92f0e8b75f7ceba7)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:22.651458 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bcecfb-3883-4a57-9a70-3cc4a5273141 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:22.656378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a411b4-de00-4664-b1f1-749f2e05f9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:22.657108 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a411b4-de00-4664-b1f1-749f2e05f9f5 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:22.657390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a411b4-de00-4664-b1f1-749f2e05f9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:22.657662 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a411b4-de00-4664-b1f1-749f2e05f9f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:22.660528 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a411b4-de00-4664-b1f1-749f2e05f9f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:22.662720 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a411b4-de00-4664-b1f1-749f2e05f9f5 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:22.715634 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 847/1678] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:22 2022] POST /identity/v3/projects => generated 393 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:22.720194 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a411b4-de00-4664-b1f1-749f2e05f9f5 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:22.725752 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-89a411b4-de00-4664-b1f1-749f2e05f9f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:22.729732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e1bcecfb-3883-4a57-9a70-3cc4a5273141 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6faaad80998e4fae92f0e8b75f7ceba7'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:22.746397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e1bcecfb-3883-4a57-9a70-3cc4a5273141 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6faaad80998e4fae92f0e8b75f7ceba7'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:22.760383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e1bcecfb-3883-4a57-9a70-3cc4a5273141 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6faaad80998e4fae92f0e8b75f7ceba7'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:22.761315 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e1bcecfb-3883-4a57-9a70-3cc4a5273141 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6faaad80998e4fae92f0e8b75f7ceba7'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:22.762082 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e1bcecfb-3883-4a57-9a70-3cc4a5273141 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6faaad80998e4fae92f0e8b75f7ceba7'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:22.815164 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 832/1679] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:47:22 2022] DELETE /identity/v3/users/6faaad80998e4fae92f0e8b75f7ceba7 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:22.848613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a411b4-de00-4664-b1f1-749f2e05f9f5 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:22.976273 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ca205-4187-44ef-a759-97d37180f8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:22.977251 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ca205-4187-44ef-a759-97d37180f8d3 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:22.977599 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ca205-4187-44ef-a759-97d37180f8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:22.977937 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ca205-4187-44ef-a759-97d37180f8d3 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:22.978710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ca205-4187-44ef-a759-97d37180f8d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:22.984008 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ca205-4187-44ef-a759-97d37180f8d3 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:23.029154 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 848/1680] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:22 2022] POST /identity/v3/users => generated 303 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:23.030000 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bcecfb-3883-4a57-9a70-3cc4a5273141 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:23.031553 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e1bcecfb-3883-4a57-9a70-3cc4a5273141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:23.046693 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bcecfb-3883-4a57-9a70-3cc4a5273141 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:23.056544 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b1f94-279e-4da4-8419-8b66fdd09b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:23.057288 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b1f94-279e-4da4-8419-8b66fdd09b8c admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:23.057564 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b1f94-279e-4da4-8419-8b66fdd09b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:23.057832 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b1f94-279e-4da4-8419-8b66fdd09b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:23.059337 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b1f94-279e-4da4-8419-8b66fdd09b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:23.060997 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b1f94-279e-4da4-8419-8b66fdd09b8c admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:23.094603 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 833/1681] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:47:23.099747 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ca205-4187-44ef-a759-97d37180f8d3 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:23.117138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ca205-4187-44ef-a759-97d37180f8d3 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:23.124817 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea788617-33df-4360-ba13-3ebb11160080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:23.125822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea788617-33df-4360-ba13-3ebb11160080 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:23.126089 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea788617-33df-4360-ba13-3ebb11160080 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:23.126359 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea788617-33df-4360-ba13-3ebb11160080 admin admin] PATH_INFO: `/v3/projects/da3442de7b7644d7a34e5348cc3ced10/users/41599522fd3a4beab6f709858c204e5b/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:23.146384 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea788617-33df-4360-ba13-3ebb11160080 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da3442de7b7644d7a34e5348cc3ced10, user_id=41599522fd3a4beab6f709858c204e5b, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:23.152281 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea788617-33df-4360-ba13-3ebb11160080 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:23.189324 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 849/1682] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:23 2022] PUT /identity/v3/projects/da3442de7b7644d7a34e5348cc3ced10/users/41599522fd3a4beab6f709858c204e5b/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:23.190447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b1f94-279e-4da4-8419-8b66fdd09b8c admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:23.204123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b1f94-279e-4da4-8419-8b66fdd09b8c admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:23.342236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5337b64-d1d0-46a6-85de-e74ce6f9b42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:23.343104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5337b64-d1d0-46a6-85de-e74ce6f9b42a admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:23.343579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5337b64-d1d0-46a6-85de-e74ce6f9b42a admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:23.344013 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5337b64-d1d0-46a6-85de-e74ce6f9b42a admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:23.345272 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5337b64-d1d0-46a6-85de-e74ce6f9b42a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:23.350870 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5337b64-d1d0-46a6-85de-e74ce6f9b42a admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:23.379171 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 834/1683] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:47:23.383364 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea788617-33df-4360-ba13-3ebb11160080 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:23.395859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea788617-33df-4360-ba13-3ebb11160080 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:23.414761 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb55fc-03ed-48bb-b60f-173165ef1c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:23.415683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb55fc-03ed-48bb-b60f-173165ef1c6b admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:23.415967 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb55fc-03ed-48bb-b60f-173165ef1c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:23.416235 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb55fc-03ed-48bb-b60f-173165ef1c6b admin admin] PATH_INFO: `/v3/projects/da3442de7b7644d7a34e5348cc3ced10/users/41599522fd3a4beab6f709858c204e5b/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:23.418293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb55fc-03ed-48bb-b60f-173165ef1c6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=da3442de7b7644d7a34e5348cc3ced10, user_id=41599522fd3a4beab6f709858c204e5b, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:23.421027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb55fc-03ed-48bb-b60f-173165ef1c6b admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:23.460836 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 850/1684] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:23 2022] PUT /identity/v3/projects/da3442de7b7644d7a34e5348cc3ced10/users/41599522fd3a4beab6f709858c204e5b/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:23.463885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7974451-e4cb-478a-946d-5139d0c1a2cb None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:23.464238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7974451-e4cb-478a-946d-5139d0c1a2cb None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:23.464573 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7974451-e4cb-478a-946d-5139d0c1a2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:23.536501 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-f7974451-e4cb-478a-946d-5139d0c1a2cb None None] MFA Rules not processed for user `41599522fd3a4beab6f709858c204e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:23.537945 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f7974451-e4cb-478a-946d-5139d0c1a2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:23.777227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 835/1685] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:23 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:23.788853 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb55fc-03ed-48bb-b60f-173165ef1c6b admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:23.790679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-0adb55fc-03ed-48bb-b60f-173165ef1c6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:23.976681 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7974451-e4cb-478a-946d-5139d0c1a2cb None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:23.978461 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f7974451-e4cb-478a-946d-5139d0c1a2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:24.044251 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb55fc-03ed-48bb-b60f-173165ef1c6b admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:24.057686 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca9176-8aaa-4d8a-966a-128b3d90bdf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:24.059813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca9176-8aaa-4d8a-966a-128b3d90bdf0 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:24.062043 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca9176-8aaa-4d8a-966a-128b3d90bdf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:24.062423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca9176-8aaa-4d8a-966a-128b3d90bdf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:24.099286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca9176-8aaa-4d8a-966a-128b3d90bdf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:24.104008 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca9176-8aaa-4d8a-966a-128b3d90bdf0 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:24.139131 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 851/1686] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:24.142382 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c8288-949e-489c-9e3a-5cf919222113 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:24.144073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c8288-949e-489c-9e3a-5cf919222113 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:24.144166 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c8288-949e-489c-9e3a-5cf919222113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:24.175952 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7974451-e4cb-478a-946d-5139d0c1a2cb None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:24.187269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c205b-a1fe-43f9-9efe-7fcbb541b4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:24.188280 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c205b-a1fe-43f9-9efe-7fcbb541b4c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:24.188463 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c205b-a1fe-43f9-9efe-7fcbb541b4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:24.190852 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c205b-a1fe-43f9-9efe-7fcbb541b4c6 admin admin] PATH_INFO: `/v3/projects/67065741880948658178d1136b4d9d67` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:24.193398 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c205b-a1fe-43f9-9efe-7fcbb541b4c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67065741880948658178d1136b4d9d67)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:24.195486 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c205b-a1fe-43f9-9efe-7fcbb541b4c6 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:24.203984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-859c8288-949e-489c-9e3a-5cf919222113 None None] MFA Rules not processed for user `76dc34045544496ab030864c5a0ffbff`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:24.205309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-859c8288-949e-489c-9e3a-5cf919222113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:24.299314 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-049c205b-a1fe-43f9-9efe-7fcbb541b4c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67065741880948658178d1136b4d9d67', 'request_id': 'req-049c205b-a1fe-43f9-9efe-7fcbb541b4c6'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:24.315078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-049c205b-a1fe-43f9-9efe-7fcbb541b4c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67065741880948658178d1136b4d9d67', 'request_id': 'req-049c205b-a1fe-43f9-9efe-7fcbb541b4c6'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:24.328317 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-049c205b-a1fe-43f9-9efe-7fcbb541b4c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67065741880948658178d1136b4d9d67', 'request_id': 'req-049c205b-a1fe-43f9-9efe-7fcbb541b4c6'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:24.357800 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-049c205b-a1fe-43f9-9efe-7fcbb541b4c6 admin admin] The token cache is being invalidate because project 67065741880948658178d1136b4d9d67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:47:24.358207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-049c205b-a1fe-43f9-9efe-7fcbb541b4c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:24.358879 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-049c205b-a1fe-43f9-9efe-7fcbb541b4c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:24.362023 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 852/1688] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:24.364771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c8288-949e-489c-9e3a-5cf919222113 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:24.365545 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 836/1688] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:47:23 2022] DELETE /identity/v3/projects/67065741880948658178d1136b4d9d67 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:24.366455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-859c8288-949e-489c-9e3a-5cf919222113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:24.368966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccd881-5c55-4026-a3dd-2d89c10a472a None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:24.369518 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccd881-5c55-4026-a3dd-2d89c10a472a None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:24.369974 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccd881-5c55-4026-a3dd-2d89c10a472a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:24.400875 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c8288-949e-489c-9e3a-5cf919222113 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:24.410104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-f7ccd881-5c55-4026-a3dd-2d89c10a472a None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:24.412826 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f7ccd881-5c55-4026-a3dd-2d89c10a472a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:24.515722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2e63b-8d87-455e-9e69-7042f494a3e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:24.516493 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2e63b-8d87-455e-9e69-7042f494a3e6 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:24.516862 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2e63b-8d87-455e-9e69-7042f494a3e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:24.517224 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2e63b-8d87-455e-9e69-7042f494a3e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:24.521049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-27a2e63b-8d87-455e-9e69-7042f494a3e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:24.535560 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 837/1689] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:47:24.536759 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccd881-5c55-4026-a3dd-2d89c10a472a None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:24.544513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f7ccd881-5c55-4026-a3dd-2d89c10a472a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:24.692188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccd881-5c55-4026-a3dd-2d89c10a472a None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:24.698469 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73442253-1528-4438-a5fa-6592f3711844 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:24.699682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73442253-1528-4438-a5fa-6592f3711844 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:24.700637 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73442253-1528-4438-a5fa-6592f3711844 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:24.701088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73442253-1528-4438-a5fa-6592f3711844 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:24.703269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-73442253-1528-4438-a5fa-6592f3711844 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:24.703467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2e63b-8d87-455e-9e69-7042f494a3e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:24.707456 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2e63b-8d87-455e-9e69-7042f494a3e6 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:24.740058 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 853/1690] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:24.743641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb479e1-e577-45fe-869a-460df8b11296 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:24.744250 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb479e1-e577-45fe-869a-460df8b11296 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:24.744250 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb479e1-e577-45fe-869a-460df8b11296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:24.783356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-0fb479e1-e577-45fe-869a-460df8b11296 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:24.786904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-0fb479e1-e577-45fe-869a-460df8b11296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:24.800744 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 854/1691] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:24.802216 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb479e1-e577-45fe-869a-460df8b11296 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:24.823929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb479e1-e577-45fe-869a-460df8b11296 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:24.832540 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73442253-1528-4438-a5fa-6592f3711844 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:24.835805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73442253-1528-4438-a5fa-6592f3711844 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:24.846283 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818d857-719a-4cc6-8539-859239b1478a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:24.847319 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 838/1692] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:24.861215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818d857-719a-4cc6-8539-859239b1478a admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:24.861901 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818d857-719a-4cc6-8539-859239b1478a admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:24.862454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818d857-719a-4cc6-8539-859239b1478a admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:24.862971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818d857-719a-4cc6-8539-859239b1478a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:47:24.863275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818d857-719a-4cc6-8539-859239b1478a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:24.869005 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818d857-719a-4cc6-8539-859239b1478a admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:24.910406 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 855/1693] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:47:24.914883 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73442253-1528-4438-a5fa-6592f3711844 service nova] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:24.929593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73442253-1528-4438-a5fa-6592f3711844 service nova] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:24.936578 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c064d-366a-4cc4-bd28-de81c1fa0b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:24.937648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c064d-366a-4cc4-bd28-de81c1fa0b5f admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:24.938281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c064d-366a-4cc4-bd28-de81c1fa0b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:24.938756 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c064d-366a-4cc4-bd28-de81c1fa0b5f admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:24.940044 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c064d-366a-4cc4-bd28-de81c1fa0b5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:24.944348 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c064d-366a-4cc4-bd28-de81c1fa0b5f admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:24.996160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 839/1694] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:24 2022] POST /identity/v3/projects => generated 369 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:24.998641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818d857-719a-4cc6-8539-859239b1478a admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:25.014750 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818d857-719a-4cc6-8539-859239b1478a admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:25.190236 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2ac30-569c-4e9a-8d66-34499ebefcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:25.191447 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2ac30-569c-4e9a-8d66-34499ebefcd6 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:25.191756 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2ac30-569c-4e9a-8d66-34499ebefcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:25.192027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2ac30-569c-4e9a-8d66-34499ebefcd6 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:25.192763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2ac30-569c-4e9a-8d66-34499ebefcd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:25.195594 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2ac30-569c-4e9a-8d66-34499ebefcd6 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:25.241851 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c064d-366a-4cc4-bd28-de81c1fa0b5f admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:25.242278 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 856/1695] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:24 2022] POST /identity/v3/users => generated 285 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:47:25.256698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c064d-366a-4cc4-bd28-de81c1fa0b5f admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:25.272293 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07195638-fce5-459d-87f9-4637a2f614c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:25.276493 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07195638-fce5-459d-87f9-4637a2f614c6 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:25.276976 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07195638-fce5-459d-87f9-4637a2f614c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:25.277421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07195638-fce5-459d-87f9-4637a2f614c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:25.278347 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07195638-fce5-459d-87f9-4637a2f614c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:25.290405 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07195638-fce5-459d-87f9-4637a2f614c6 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:25.326976 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 840/1696] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:25.331462 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2ac30-569c-4e9a-8d66-34499ebefcd6 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:25.352301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2ac30-569c-4e9a-8d66-34499ebefcd6 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:25.378683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf28bc-21f1-4eaa-855f-6209f7d762ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:25.379386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf28bc-21f1-4eaa-855f-6209f7d762ec admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:25.382862 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf28bc-21f1-4eaa-855f-6209f7d762ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:25.383186 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf28bc-21f1-4eaa-855f-6209f7d762ec admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:25.383962 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf28bc-21f1-4eaa-855f-6209f7d762ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:25.393779 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf28bc-21f1-4eaa-855f-6209f7d762ec admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:25.440346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 857/1697] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:25.444986 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07195638-fce5-459d-87f9-4637a2f614c6 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:25.460453 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07195638-fce5-459d-87f9-4637a2f614c6 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:25.492588 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b522b8-52a8-4796-8180-dce6aaeb8e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:25.496093 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b522b8-52a8-4796-8180-dce6aaeb8e1f admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:25.496462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b522b8-52a8-4796-8180-dce6aaeb8e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:25.496636 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b522b8-52a8-4796-8180-dce6aaeb8e1f admin admin] PATH_INFO: `/v3/projects/b8967945bae54fcea337db7ac4904aad/users/549aee398b3542eb9d0ede4f43be7f54/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:25.564914 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b522b8-52a8-4796-8180-dce6aaeb8e1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8967945bae54fcea337db7ac4904aad, user_id=549aee398b3542eb9d0ede4f43be7f54, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:25.568756 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b522b8-52a8-4796-8180-dce6aaeb8e1f admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:25.595979 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 841/1698] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:25 2022] PUT /identity/v3/projects/b8967945bae54fcea337db7ac4904aad/users/549aee398b3542eb9d0ede4f43be7f54/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:25.606728 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850bd36-e71f-4312-a34c-9a50ced9596d None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:25.609053 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850bd36-e71f-4312-a34c-9a50ced9596d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:25.612117 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850bd36-e71f-4312-a34c-9a50ced9596d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:25.706638 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-6850bd36-e71f-4312-a34c-9a50ced9596d None None] MFA Rules not processed for user `549aee398b3542eb9d0ede4f43be7f54`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:25.707878 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-6850bd36-e71f-4312-a34c-9a50ced9596d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:25.866241 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 858/1699] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:25 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:25.872055 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b522b8-52a8-4796-8180-dce6aaeb8e1f admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:26.037454 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b522b8-52a8-4796-8180-dce6aaeb8e1f admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:26.045559 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4ce69-9bc8-4b55-b20d-36d5600b86f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:26.046438 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4ce69-9bc8-4b55-b20d-36d5600b86f1 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:26.047484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4ce69-9bc8-4b55-b20d-36d5600b86f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:26.048052 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4ce69-9bc8-4b55-b20d-36d5600b86f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:26.049271 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4ce69-9bc8-4b55-b20d-36d5600b86f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:26.052421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4ce69-9bc8-4b55-b20d-36d5600b86f1 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:26.104914 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850bd36-e71f-4312-a34c-9a50ced9596d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:26.108477 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 842/1700] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:25 2022] POST /identity/v3/projects => generated 369 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:26.123883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850bd36-e71f-4312-a34c-9a50ced9596d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:26.166015 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4ce69-9bc8-4b55-b20d-36d5600b86f1 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:26.169250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-c2c4ce69-9bc8-4b55-b20d-36d5600b86f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:26.280709 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba73f689-5bca-45e0-8c01-0bb91ffd70ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:26.281481 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73f689-5bca-45e0-8c01-0bb91ffd70ca admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:26.281765 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73f689-5bca-45e0-8c01-0bb91ffd70ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:26.282038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73f689-5bca-45e0-8c01-0bb91ffd70ca admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:26.282920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba73f689-5bca-45e0-8c01-0bb91ffd70ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:26.287457 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba73f689-5bca-45e0-8c01-0bb91ffd70ca admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:26.321043 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 859/1701] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:26 2022] POST /identity/v3/users => generated 291 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:47:26.324900 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba73f689-5bca-45e0-8c01-0bb91ffd70ca admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:26.344082 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4ce69-9bc8-4b55-b20d-36d5600b86f1 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:26.349586 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775be820-8477-48cc-9cc5-9ca0c934107b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:26.349800 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba73f689-5bca-45e0-8c01-0bb91ffd70ca admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:26.350261 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775be820-8477-48cc-9cc5-9ca0c934107b service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:26.350636 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775be820-8477-48cc-9cc5-9ca0c934107b service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:26.352061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775be820-8477-48cc-9cc5-9ca0c934107b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:26.357524 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-775be820-8477-48cc-9cc5-9ca0c934107b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:26.362301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb3960-c0d5-4aa3-83e7-6f86a865a9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:26.363141 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb3960-c0d5-4aa3-83e7-6f86a865a9df admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:26.363520 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb3960-c0d5-4aa3-83e7-6f86a865a9df admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:26.363921 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb3960-c0d5-4aa3-83e7-6f86a865a9df admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:26.365431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb3960-c0d5-4aa3-83e7-6f86a865a9df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:26.367811 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb3960-c0d5-4aa3-83e7-6f86a865a9df admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:26.389110 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 860/1702] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:26 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:47:26.393472 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb3960-c0d5-4aa3-83e7-6f86a865a9df admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:26.419034 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb3960-c0d5-4aa3-83e7-6f86a865a9df admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:26.431634 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f56e2f-f446-444f-8e5d-cee1a393d6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:26.432728 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f56e2f-f446-444f-8e5d-cee1a393d6fd admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:26.433125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f56e2f-f446-444f-8e5d-cee1a393d6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:26.433494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f56e2f-f446-444f-8e5d-cee1a393d6fd admin admin] PATH_INFO: `/v3/projects/85a582b3c1fa4e2f9c2255d967108963/users/8af4488eaef44e1f9667c39e09ed1f67/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:26.470130 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775be820-8477-48cc-9cc5-9ca0c934107b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:26.475783 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775be820-8477-48cc-9cc5-9ca0c934107b service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:26.496365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f56e2f-f446-444f-8e5d-cee1a393d6fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a582b3c1fa4e2f9c2255d967108963, user_id=8af4488eaef44e1f9667c39e09ed1f67, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:26.502877 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f56e2f-f446-444f-8e5d-cee1a393d6fd admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:26.509190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 843/1703] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:26.524472 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 861/1704] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:26 2022] PUT /identity/v3/projects/85a582b3c1fa4e2f9c2255d967108963/users/8af4488eaef44e1f9667c39e09ed1f67/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:26.528926 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775be820-8477-48cc-9cc5-9ca0c934107b service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:26.546041 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775be820-8477-48cc-9cc5-9ca0c934107b service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:26.685641 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769ff5a-b4e3-429b-9d23-6ec87040e5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:26.686338 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769ff5a-b4e3-429b-9d23-6ec87040e5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:26.686625 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769ff5a-b4e3-429b-9d23-6ec87040e5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:26.686888 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769ff5a-b4e3-429b-9d23-6ec87040e5e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:26.687684 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769ff5a-b4e3-429b-9d23-6ec87040e5e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:26.694102 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769ff5a-b4e3-429b-9d23-6ec87040e5e6 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:26.720331 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f56e2f-f446-444f-8e5d-cee1a393d6fd admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:26.727508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 844/1705] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:26 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:47:26.735444 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f56e2f-f446-444f-8e5d-cee1a393d6fd admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:26.769622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd673d-42ed-436d-b548-86c69f6cf2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:26.773309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd673d-42ed-436d-b548-86c69f6cf2f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:26.774682 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd673d-42ed-436d-b548-86c69f6cf2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:26.776257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd673d-42ed-436d-b548-86c69f6cf2f0 admin admin] PATH_INFO: `/v3/projects/85a582b3c1fa4e2f9c2255d967108963/users/8af4488eaef44e1f9667c39e09ed1f67/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:26.782355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd673d-42ed-436d-b548-86c69f6cf2f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a582b3c1fa4e2f9c2255d967108963, user_id=8af4488eaef44e1f9667c39e09ed1f67, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:26.785965 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd673d-42ed-436d-b548-86c69f6cf2f0 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:26.802450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 862/1706] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:26 2022] PUT /identity/v3/projects/85a582b3c1fa4e2f9c2255d967108963/users/8af4488eaef44e1f9667c39e09ed1f67/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:26.815170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac6474-e110-4834-b55c-be26c4be91ba None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:26.815450 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac6474-e110-4834-b55c-be26c4be91ba None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:26.815882 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac6474-e110-4834-b55c-be26c4be91ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:26.865152 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-e6ac6474-e110-4834-b55c-be26c4be91ba None None] MFA Rules not processed for user `8af4488eaef44e1f9667c39e09ed1f67`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:26.866963 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e6ac6474-e110-4834-b55c-be26c4be91ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:27.032147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 845/1707] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:26 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:27.045091 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd673d-42ed-436d-b548-86c69f6cf2f0 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:27.047161 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-4abd673d-42ed-436d-b548-86c69f6cf2f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:27.151899 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd673d-42ed-436d-b548-86c69f6cf2f0 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:27.174099 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76ba6a-b101-4143-bc4b-bb28662e2984 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:27.175036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76ba6a-b101-4143-bc4b-bb28662e2984 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:27.175471 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76ba6a-b101-4143-bc4b-bb28662e2984 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:27.177351 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76ba6a-b101-4143-bc4b-bb28662e2984 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:27.217485 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76ba6a-b101-4143-bc4b-bb28662e2984 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:27.222340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76ba6a-b101-4143-bc4b-bb28662e2984 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:27.244152 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 863/1708] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:27 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:28.202114 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43481ea-1909-47f3-899c-64d04fb6b1a2 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:28.202641 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43481ea-1909-47f3-899c-64d04fb6b1a2 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:28.202641 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43481ea-1909-47f3-899c-64d04fb6b1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:28.269608 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-b43481ea-1909-47f3-899c-64d04fb6b1a2 None None] MFA Rules not processed for user `e58c14dca7da4e6495f7e08f2bb127e2`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:28.280014 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-b43481ea-1909-47f3-899c-64d04fb6b1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:28.480122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 846/1709] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:28.491514 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b600b6-c3ad-4d14-a090-73da9ae12a84 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:28.492637 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b600b6-c3ad-4d14-a090-73da9ae12a84 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:28.493188 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b600b6-c3ad-4d14-a090-73da9ae12a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:28.528878 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e969e-9f81-46b6-8a4d-cae9088ab83a None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:28.529167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e969e-9f81-46b6-8a4d-cae9088ab83a None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:28.529436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e969e-9f81-46b6-8a4d-cae9088ab83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:28.544572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-e0b600b6-c3ad-4d14-a090-73da9ae12a84 None None] MFA Rules not processed for user `e58c14dca7da4e6495f7e08f2bb127e2`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:28.548024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e0b600b6-c3ad-4d14-a090-73da9ae12a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:28.569422 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-151e969e-9f81-46b6-8a4d-cae9088ab83a None None] MFA Rules not processed for user `76dc34045544496ab030864c5a0ffbff`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:28.573340 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-151e969e-9f81-46b6-8a4d-cae9088ab83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:28.580614 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 864/1710] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:28.598598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b600b6-c3ad-4d14-a090-73da9ae12a84 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:28.712783 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 847/1711] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:47:28.821471 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b600b6-c3ad-4d14-a090-73da9ae12a84 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:28.837496 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff3022-a79b-40e5-8029-0a2f3cec2595 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:28.838407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff3022-a79b-40e5-8029-0a2f3cec2595 service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:28.838689 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff3022-a79b-40e5-8029-0a2f3cec2595 service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:28.840135 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff3022-a79b-40e5-8029-0a2f3cec2595 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:28.865115 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff3022-a79b-40e5-8029-0a2f3cec2595 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:28.871193 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff3022-a79b-40e5-8029-0a2f3cec2595 service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:28.911090 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 865/1712] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:29.055873 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997a69e-9a41-4a23-947c-c9ed3d811bc6 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:29.056157 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997a69e-9a41-4a23-947c-c9ed3d811bc6 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:29.056515 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997a69e-9a41-4a23-947c-c9ed3d811bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:29.109875 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-4997a69e-9a41-4a23-947c-c9ed3d811bc6 None None] MFA Rules not processed for user `f89c5ab1020947a9bece03af4c237480`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:29.111608 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-4997a69e-9a41-4a23-947c-c9ed3d811bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:29.399938 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 848/1713] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:29.838523 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f08291-4797-4c2f-a7e8-223bcc594fab None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:29.839508 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f08291-4797-4c2f-a7e8-223bcc594fab None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:29.839914 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f08291-4797-4c2f-a7e8-223bcc594fab None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:29.841511 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 866/1714] 10.222.0.26 () {54 vars in 912 bytes} [Mon May 9 09:47:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:47:29.849727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a5ff9-3754-460a-bdf9-8593ff75a3d5 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:29.850303 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a5ff9-3754-460a-bdf9-8593ff75a3d5 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:29.850787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a5ff9-3754-460a-bdf9-8593ff75a3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:30.085484 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f08291-4797-4c2f-a7e8-223bcc594fab None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:30.193693 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-474a5ff9-3754-460a-bdf9-8593ff75a3d5 None None] MFA Rules not processed for user `0f0d3f4264a04093894d6bc5c5c45c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:30.197410 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-474a5ff9-3754-460a-bdf9-8593ff75a3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:30.239457 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f08291-4797-4c2f-a7e8-223bcc594fab None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:30.244574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ea8e1-0435-4939-a00b-44009ba78273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:30.245793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ea8e1-0435-4939-a00b-44009ba78273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:30.246259 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ea8e1-0435-4939-a00b-44009ba78273 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:30.246720 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ea8e1-0435-4939-a00b-44009ba78273 admin admin] PATH_INFO: `/v3/users/549aee398b3542eb9d0ede4f43be7f54` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:30.248146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ea8e1-0435-4939-a00b-44009ba78273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=549aee398b3542eb9d0ede4f43be7f54)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:30.252255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ea8e1-0435-4939-a00b-44009ba78273 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:30.357018 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 849/1715] 10.222.0.26 () {58 vars in 1009 bytes} [Mon May 9 09:47:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 508 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:30.360399 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a5ff9-3754-460a-bdf9-8593ff75a3d5 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:30.362574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-600ea8e1-0435-4939-a00b-44009ba78273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '549aee398b3542eb9d0ede4f43be7f54'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:30.380482 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-600ea8e1-0435-4939-a00b-44009ba78273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '549aee398b3542eb9d0ede4f43be7f54'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:30.383529 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-600ea8e1-0435-4939-a00b-44009ba78273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '549aee398b3542eb9d0ede4f43be7f54'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:30.393285 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a5ff9-3754-460a-bdf9-8593ff75a3d5 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:30.411564 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-600ea8e1-0435-4939-a00b-44009ba78273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '549aee398b3542eb9d0ede4f43be7f54'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:30.412277 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-600ea8e1-0435-4939-a00b-44009ba78273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '549aee398b3542eb9d0ede4f43be7f54'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:30.427727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48137f58-41ac-4f08-973d-ab5391a8b633 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:30.429744 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48137f58-41ac-4f08-973d-ab5391a8b633 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:30.430424 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48137f58-41ac-4f08-973d-ab5391a8b633 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:30.430955 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48137f58-41ac-4f08-973d-ab5391a8b633 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:30.434855 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-48137f58-41ac-4f08-973d-ab5391a8b633 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:30.461293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 867/1716] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:47:30 2022] DELETE /identity/v3/users/549aee398b3542eb9d0ede4f43be7f54 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:47:30.462502 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ea8e1-0435-4939-a00b-44009ba78273 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:30.465866 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-600ea8e1-0435-4939-a00b-44009ba78273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:30.467548 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48137f58-41ac-4f08-973d-ab5391a8b633 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:30.469516 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48137f58-41ac-4f08-973d-ab5391a8b633 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:30.483239 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 850/1717] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:30.485034 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e457da-f05c-471c-848a-3bda53461954 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:30.485381 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e457da-f05c-471c-848a-3bda53461954 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:30.485924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e457da-f05c-471c-848a-3bda53461954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:30.506155 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-72e457da-f05c-471c-848a-3bda53461954 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:30.507528 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-72e457da-f05c-471c-848a-3bda53461954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:30.624922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ea8e1-0435-4939-a00b-44009ba78273 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:30.633582 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a848b-acf1-4f7e-950b-41ac5d27706a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:30.634078 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a848b-acf1-4f7e-950b-41ac5d27706a service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:30.634414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a848b-acf1-4f7e-950b-41ac5d27706a service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:30.634607 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a848b-acf1-4f7e-950b-41ac5d27706a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:30.638206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-ec4a848b-acf1-4f7e-950b-41ac5d27706a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:30.661177 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a848b-acf1-4f7e-950b-41ac5d27706a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:30.663281 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a848b-acf1-4f7e-950b-41ac5d27706a service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:30.668137 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 851/1718] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:30.677760 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e457da-f05c-471c-848a-3bda53461954 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:30.689999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 868/1719] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:30.709124 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e457da-f05c-471c-848a-3bda53461954 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:30.720578 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7da43b-1864-4f32-9907-05e9b1f82f70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:30.721740 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7da43b-1864-4f32-9907-05e9b1f82f70 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:30.722018 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7da43b-1864-4f32-9907-05e9b1f82f70 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:30.722392 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7da43b-1864-4f32-9907-05e9b1f82f70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:30.741996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7da43b-1864-4f32-9907-05e9b1f82f70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:30.743454 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7da43b-1864-4f32-9907-05e9b1f82f70 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:30.769175 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 852/1720] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 09:47:30.823385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc0cb1-c2fa-4237-952b-5a447503764f None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:30.823666 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc0cb1-c2fa-4237-952b-5a447503764f None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:30.823931 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc0cb1-c2fa-4237-952b-5a447503764f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:30.870441 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-67cc0cb1-c2fa-4237-952b-5a447503764f None None] MFA Rules not processed for user `af62b2d3f73f482b8db4173af33752c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:30.874644 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-67cc0cb1-c2fa-4237-952b-5a447503764f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:31.113329 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 869/1721] 10.222.0.26 () {58 vars in 1009 bytes} [Mon May 9 09:47:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:31.520638 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7da43b-1864-4f32-9907-05e9b1f82f70 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:31.526433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-3f7da43b-1864-4f32-9907-05e9b1f82f70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:31.701867 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7da43b-1864-4f32-9907-05e9b1f82f70 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:31.715919 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e955001-0eda-49ff-a60f-fcb68913d048 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380e04358f2349aa912d6a97518877cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:31.716786 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e955001-0eda-49ff-a60f-fcb68913d048 service swift] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:31.717337 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e955001-0eda-49ff-a60f-fcb68913d048 service swift] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:31.717747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e955001-0eda-49ff-a60f-fcb68913d048 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:31.740329 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e955001-0eda-49ff-a60f-fcb68913d048 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:31.757552 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e955001-0eda-49ff-a60f-fcb68913d048 service swift] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:31.765946 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb7677-117a-4fa9-bdbe-3752c91a2857 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:31.766404 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb7677-117a-4fa9-bdbe-3752c91a2857 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:31.766935 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb7677-117a-4fa9-bdbe-3752c91a2857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:31.771010 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 853/1722] 10.222.0.26 () {60 vars in 1384 bytes} [Mon May 9 09:47:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 09:47:31.820415 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-73cb7677-117a-4fa9-bdbe-3752c91a2857 None None] MFA Rules not processed for user `e58c14dca7da4e6495f7e08f2bb127e2`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:31.821706 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-73cb7677-117a-4fa9-bdbe-3752c91a2857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:32.162527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 870/1723] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:32.340021 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e955001-0eda-49ff-a60f-fcb68913d048 service swift] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:32.341921 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-7e955001-0eda-49ff-a60f-fcb68913d048 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:32.382200 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e955001-0eda-49ff-a60f-fcb68913d048 service swift] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:32.401362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a823fe2-510f-411b-9ece-29826a258527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:32.402549 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a823fe2-510f-411b-9ece-29826a258527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:32.402927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a823fe2-510f-411b-9ece-29826a258527 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:32.403291 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a823fe2-510f-411b-9ece-29826a258527 admin admin] PATH_INFO: `/v3/projects/b8967945bae54fcea337db7ac4904aad` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:32.409109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a823fe2-510f-411b-9ece-29826a258527 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8967945bae54fcea337db7ac4904aad)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:32.411827 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a823fe2-510f-411b-9ece-29826a258527 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:32.619562 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4a823fe2-510f-411b-9ece-29826a258527 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8967945bae54fcea337db7ac4904aad', 'request_id': 'req-4a823fe2-510f-411b-9ece-29826a258527'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:32.662153 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4a823fe2-510f-411b-9ece-29826a258527 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8967945bae54fcea337db7ac4904aad', 'request_id': 'req-4a823fe2-510f-411b-9ece-29826a258527'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:32.679113 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4a823fe2-510f-411b-9ece-29826a258527 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8967945bae54fcea337db7ac4904aad', 'request_id': 'req-4a823fe2-510f-411b-9ece-29826a258527'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:32.714069 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4a823fe2-510f-411b-9ece-29826a258527 admin admin] The token cache is being invalidate because project b8967945bae54fcea337db7ac4904aad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:47:32.714661 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4a823fe2-510f-411b-9ece-29826a258527 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:32.715533 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4a823fe2-510f-411b-9ece-29826a258527 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:32.717354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 854/1724] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:47:32 2022] DELETE /identity/v3/projects/b8967945bae54fcea337db7ac4904aad => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:47:32.721764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb7677-117a-4fa9-bdbe-3752c91a2857 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:32.736447 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb7677-117a-4fa9-bdbe-3752c91a2857 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:32.977443 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42c1ce-1ad7-42a3-9968-fb72fc9202c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:32.978776 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42c1ce-1ad7-42a3-9968-fb72fc9202c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:32.980101 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42c1ce-1ad7-42a3-9968-fb72fc9202c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:32.980588 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42c1ce-1ad7-42a3-9968-fb72fc9202c3 admin admin] PATH_INFO: `/v3/users/8af4488eaef44e1f9667c39e09ed1f67` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:32.987355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42c1ce-1ad7-42a3-9968-fb72fc9202c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8af4488eaef44e1f9667c39e09ed1f67)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:32.998300 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42c1ce-1ad7-42a3-9968-fb72fc9202c3 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:33.161790 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bd42c1ce-1ad7-42a3-9968-fb72fc9202c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8af4488eaef44e1f9667c39e09ed1f67'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:33.183472 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bd42c1ce-1ad7-42a3-9968-fb72fc9202c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8af4488eaef44e1f9667c39e09ed1f67'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:33.185879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bd42c1ce-1ad7-42a3-9968-fb72fc9202c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8af4488eaef44e1f9667c39e09ed1f67'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:33.211843 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bd42c1ce-1ad7-42a3-9968-fb72fc9202c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8af4488eaef44e1f9667c39e09ed1f67'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:33.214342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bd42c1ce-1ad7-42a3-9968-fb72fc9202c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8af4488eaef44e1f9667c39e09ed1f67'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:33.271644 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 871/1725] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:47:32 2022] DELETE /identity/v3/users/8af4488eaef44e1f9667c39e09ed1f67 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:34.100370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a823fe2-510f-411b-9ece-29826a258527 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:34.102938 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-4a823fe2-510f-411b-9ece-29826a258527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:34.120922 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a823fe2-510f-411b-9ece-29826a258527 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:34.204291 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26b1da-7e8f-4ee1-9ab5-c642997af5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:34.205473 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26b1da-7e8f-4ee1-9ab5-c642997af5c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:34.205903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26b1da-7e8f-4ee1-9ab5-c642997af5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:34.206319 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26b1da-7e8f-4ee1-9ab5-c642997af5c0 admin admin] PATH_INFO: `/v3/projects/85a582b3c1fa4e2f9c2255d967108963` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:34.207684 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26b1da-7e8f-4ee1-9ab5-c642997af5c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85a582b3c1fa4e2f9c2255d967108963)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:34.209900 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26b1da-7e8f-4ee1-9ab5-c642997af5c0 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:34.340803 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-5a26b1da-7e8f-4ee1-9ab5-c642997af5c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85a582b3c1fa4e2f9c2255d967108963', 'request_id': 'req-5a26b1da-7e8f-4ee1-9ab5-c642997af5c0'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:34.355806 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-5a26b1da-7e8f-4ee1-9ab5-c642997af5c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85a582b3c1fa4e2f9c2255d967108963', 'request_id': 'req-5a26b1da-7e8f-4ee1-9ab5-c642997af5c0'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:34.369926 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-5a26b1da-7e8f-4ee1-9ab5-c642997af5c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85a582b3c1fa4e2f9c2255d967108963', 'request_id': 'req-5a26b1da-7e8f-4ee1-9ab5-c642997af5c0'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:34.399972 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-5a26b1da-7e8f-4ee1-9ab5-c642997af5c0 admin admin] The token cache is being invalidate because project 85a582b3c1fa4e2f9c2255d967108963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:47:34.402770 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-5a26b1da-7e8f-4ee1-9ab5-c642997af5c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:34.404064 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-5a26b1da-7e8f-4ee1-9ab5-c642997af5c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:34.410057 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 855/1726] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:47:34 2022] DELETE /identity/v3/projects/85a582b3c1fa4e2f9c2255d967108963 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:34.412263 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae82c91-350d-4ce7-b19c-d1ccda615183 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:34.412785 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae82c91-350d-4ce7-b19c-d1ccda615183 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:34.412867 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae82c91-350d-4ce7-b19c-d1ccda615183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:34.480182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-1ae82c91-350d-4ce7-b19c-d1ccda615183 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:34.488287 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-1ae82c91-350d-4ce7-b19c-d1ccda615183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:34.695512 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 872/1727] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:34.700090 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc43b4-907c-47b7-8232-7a5deca66585 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:34.700384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc43b4-907c-47b7-8232-7a5deca66585 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:34.700775 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc43b4-907c-47b7-8232-7a5deca66585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:34.735768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-56fc43b4-907c-47b7-8232-7a5deca66585 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:34.738192 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-56fc43b4-907c-47b7-8232-7a5deca66585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:34.752894 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 856/1728] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:34.753946 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae82c91-350d-4ce7-b19c-d1ccda615183 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:34.764250 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae82c91-350d-4ce7-b19c-d1ccda615183 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:34.778705 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d30dbeb-a561-453e-9ddc-dc4d65f8b613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:34.779476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30dbeb-a561-453e-9ddc-dc4d65f8b613 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:34.781857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30dbeb-a561-453e-9ddc-dc4d65f8b613 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:34.782626 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30dbeb-a561-453e-9ddc-dc4d65f8b613 admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:34.784111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d30dbeb-a561-453e-9ddc-dc4d65f8b613 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:47:34.785661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d30dbeb-a561-453e-9ddc-dc4d65f8b613 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:34.787590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d30dbeb-a561-453e-9ddc-dc4d65f8b613 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:34.809892 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 873/1729] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:47:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:47:34.813942 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc43b4-907c-47b7-8232-7a5deca66585 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:34.831017 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc43b4-907c-47b7-8232-7a5deca66585 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:34.851721 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e91078-a38c-468b-ad17-c573465c12e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:34.852826 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e91078-a38c-468b-ad17-c573465c12e8 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:34.853301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e91078-a38c-468b-ad17-c573465c12e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:34.853742 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e91078-a38c-468b-ad17-c573465c12e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:34.854753 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e91078-a38c-468b-ad17-c573465c12e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:34.857458 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e91078-a38c-468b-ad17-c573465c12e8 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:34.909108 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 857/1730] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:34 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:34.913289 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d30dbeb-a561-453e-9ddc-dc4d65f8b613 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:34.927950 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d30dbeb-a561-453e-9ddc-dc4d65f8b613 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:35.142873 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db940e85-f9c9-4164-9c8d-9c9d6bbb375f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:35.145539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db940e85-f9c9-4164-9c8d-9c9d6bbb375f admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:35.146078 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db940e85-f9c9-4164-9c8d-9c9d6bbb375f admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:35.146940 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db940e85-f9c9-4164-9c8d-9c9d6bbb375f admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:35.148329 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db940e85-f9c9-4164-9c8d-9c9d6bbb375f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:35.165914 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db940e85-f9c9-4164-9c8d-9c9d6bbb375f admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:35.217270 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 874/1731] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:34 2022] POST /identity/v3/users => generated 294 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:35.221495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e91078-a38c-468b-ad17-c573465c12e8 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:35.243071 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e91078-a38c-468b-ad17-c573465c12e8 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:35.267528 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d522652-9913-4818-8fef-2e79b90ef5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:35.268325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d522652-9913-4818-8fef-2e79b90ef5b2 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:35.268819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d522652-9913-4818-8fef-2e79b90ef5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:35.269189 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d522652-9913-4818-8fef-2e79b90ef5b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:35.270107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d522652-9913-4818-8fef-2e79b90ef5b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:35.271808 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d522652-9913-4818-8fef-2e79b90ef5b2 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:35.319617 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db940e85-f9c9-4164-9c8d-9c9d6bbb375f admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:35.334958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 858/1732] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:35 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:35.348803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db940e85-f9c9-4164-9c8d-9c9d6bbb375f admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:35.369533 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a561a1f-0db7-481f-bab9-cc9aea05ae63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:35.370212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a561a1f-0db7-481f-bab9-cc9aea05ae63 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:35.370658 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a561a1f-0db7-481f-bab9-cc9aea05ae63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:35.371200 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a561a1f-0db7-481f-bab9-cc9aea05ae63 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:35.372536 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a561a1f-0db7-481f-bab9-cc9aea05ae63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:35.380181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a561a1f-0db7-481f-bab9-cc9aea05ae63 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:35.406482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d522652-9913-4818-8fef-2e79b90ef5b2 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:35.424901 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 875/1733] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:35 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:35.445697 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d522652-9913-4818-8fef-2e79b90ef5b2 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:35.463733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8a81f-e2ad-47b2-a702-234f8d6cd445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:35.466243 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8a81f-e2ad-47b2-a702-234f8d6cd445 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:35.467497 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8a81f-e2ad-47b2-a702-234f8d6cd445 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:35.467927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8a81f-e2ad-47b2-a702-234f8d6cd445 admin admin] PATH_INFO: `/v3/projects/1387607ad9e14443bede2a41a717315f/users/8aa346d326624c28bd91b13d946a1afd/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:35.533059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8a81f-e2ad-47b2-a702-234f8d6cd445 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1387607ad9e14443bede2a41a717315f, user_id=8aa346d326624c28bd91b13d946a1afd, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:35.537925 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8a81f-e2ad-47b2-a702-234f8d6cd445 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:35.583651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 859/1734] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:35 2022] PUT /identity/v3/projects/1387607ad9e14443bede2a41a717315f/users/8aa346d326624c28bd91b13d946a1afd/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:35.600200 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d372c-95b2-48f6-a59c-7c0e37151191 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:35.600764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d372c-95b2-48f6-a59c-7c0e37151191 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:35.601201 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d372c-95b2-48f6-a59c-7c0e37151191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:35.642770 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac3ef3-3004-4187-b28d-3b4cb8eb57a1 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:35.643053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac3ef3-3004-4187-b28d-3b4cb8eb57a1 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:35.643323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac3ef3-3004-4187-b28d-3b4cb8eb57a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:35.703707 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-2d2d372c-95b2-48f6-a59c-7c0e37151191 None None] MFA Rules not processed for user `8aa346d326624c28bd91b13d946a1afd`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:35.706796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-93ac3ef3-3004-4187-b28d-3b4cb8eb57a1 None None] MFA Rules not processed for user `0f0d3f4264a04093894d6bc5c5c45c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:35.707239 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-2d2d372c-95b2-48f6-a59c-7c0e37151191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:35.718109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-93ac3ef3-3004-4187-b28d-3b4cb8eb57a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:35.802802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 876/1735] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:35.811660 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d372c-95b2-48f6-a59c-7c0e37151191 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:35.848294 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 860/1736] 10.222.0.50 () {58 vars in 1015 bytes} [Mon May 9 09:47:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:47:36.113421 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d372c-95b2-48f6-a59c-7c0e37151191 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:36.130095 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03140721-5e11-4b78-81cb-2016788c6fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:36.131392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03140721-5e11-4b78-81cb-2016788c6fc9 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:36.131864 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03140721-5e11-4b78-81cb-2016788c6fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:36.132276 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03140721-5e11-4b78-81cb-2016788c6fc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:36.133227 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03140721-5e11-4b78-81cb-2016788c6fc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:36.136727 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03140721-5e11-4b78-81cb-2016788c6fc9 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:36.264159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 877/1737] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:35 2022] POST /identity/v3/projects => generated 387 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:36.268586 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac3ef3-3004-4187-b28d-3b4cb8eb57a1 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:36.285808 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac3ef3-3004-4187-b28d-3b4cb8eb57a1 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:36.433415 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7811ef-fe22-4824-b921-44f6f98b2590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:36.434258 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7811ef-fe22-4824-b921-44f6f98b2590 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:36.434513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7811ef-fe22-4824-b921-44f6f98b2590 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:36.434952 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7811ef-fe22-4824-b921-44f6f98b2590 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:36.435796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7811ef-fe22-4824-b921-44f6f98b2590 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:36.441163 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7811ef-fe22-4824-b921-44f6f98b2590 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:36.487237 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 861/1738] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:36 2022] POST /identity/v3/users => generated 300 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:36.491149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03140721-5e11-4b78-81cb-2016788c6fc9 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:36.506149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03140721-5e11-4b78-81cb-2016788c6fc9 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:36.516086 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5514b6-2b25-4470-a052-13adb8c63205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:36.523233 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5514b6-2b25-4470-a052-13adb8c63205 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:36.524459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5514b6-2b25-4470-a052-13adb8c63205 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:36.524893 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5514b6-2b25-4470-a052-13adb8c63205 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:36.527211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5514b6-2b25-4470-a052-13adb8c63205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:36.530998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5514b6-2b25-4470-a052-13adb8c63205 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:36.551814 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b0a2e-57d1-4ff8-a533-b01c789d54d4 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:36.552550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b0a2e-57d1-4ff8-a533-b01c789d54d4 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:36.552996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b0a2e-57d1-4ff8-a533-b01c789d54d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:36.586378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 878/1739] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:47:36.589180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5514b6-2b25-4470-a052-13adb8c63205 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:36.589862 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-691b0a2e-57d1-4ff8-a533-b01c789d54d4 None None] MFA Rules not processed for user `f89c5ab1020947a9bece03af4c237480`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:36.591864 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-691b0a2e-57d1-4ff8-a533-b01c789d54d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:36.600395 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5514b6-2b25-4470-a052-13adb8c63205 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:36.607273 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059eacc-beec-4a24-987d-392a4889fe67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:36.609088 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059eacc-beec-4a24-987d-392a4889fe67 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:36.610325 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059eacc-beec-4a24-987d-392a4889fe67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:36.610931 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059eacc-beec-4a24-987d-392a4889fe67 admin admin] PATH_INFO: `/v3/projects/3ea9ffde7af44e6a9e1b6c5c79232eb0/users/5a0fd07d66ae48a9b46f6a2ce9996326/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:36.638712 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059eacc-beec-4a24-987d-392a4889fe67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ea9ffde7af44e6a9e1b6c5c79232eb0, user_id=5a0fd07d66ae48a9b46f6a2ce9996326, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:36.643246 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059eacc-beec-4a24-987d-392a4889fe67 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:36.691732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 879/1740] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:36 2022] PUT /identity/v3/projects/3ea9ffde7af44e6a9e1b6c5c79232eb0/users/5a0fd07d66ae48a9b46f6a2ce9996326/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:36.699046 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059eacc-beec-4a24-987d-392a4889fe67 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:36.755035 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 862/1741] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:36.915297 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059eacc-beec-4a24-987d-392a4889fe67 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:36.927703 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157a893-d1f7-4a43-844a-51901addba50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:36.928814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157a893-d1f7-4a43-844a-51901addba50 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:36.929245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157a893-d1f7-4a43-844a-51901addba50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:36.929529 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157a893-d1f7-4a43-844a-51901addba50 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:36.937367 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157a893-d1f7-4a43-844a-51901addba50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:36.939034 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157a893-d1f7-4a43-844a-51901addba50 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:36.970843 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 880/1742] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:36.971539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b0a2e-57d1-4ff8-a533-b01c789d54d4 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:36.990795 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b0a2e-57d1-4ff8-a533-b01c789d54d4 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:37.019031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0631b2-948a-4e3a-95d1-95b3c08d74ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:37.019992 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0631b2-948a-4e3a-95d1-95b3c08d74ec admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:37.020229 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0631b2-948a-4e3a-95d1-95b3c08d74ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:37.020632 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0631b2-948a-4e3a-95d1-95b3c08d74ec admin admin] PATH_INFO: `/v3/projects/3ea9ffde7af44e6a9e1b6c5c79232eb0/users/5a0fd07d66ae48a9b46f6a2ce9996326/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:37.029900 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0631b2-948a-4e3a-95d1-95b3c08d74ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ea9ffde7af44e6a9e1b6c5c79232eb0, user_id=5a0fd07d66ae48a9b46f6a2ce9996326, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:37.032626 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0631b2-948a-4e3a-95d1-95b3c08d74ec admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:37.063055 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 863/1743] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:36 2022] PUT /identity/v3/projects/3ea9ffde7af44e6a9e1b6c5c79232eb0/users/5a0fd07d66ae48a9b46f6a2ce9996326/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:47:37.074995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679515b9-c50c-411d-9bf5-10a83e50de1d None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:37.075285 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679515b9-c50c-411d-9bf5-10a83e50de1d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:37.075606 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679515b9-c50c-411d-9bf5-10a83e50de1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:37.137194 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-679515b9-c50c-411d-9bf5-10a83e50de1d None None] MFA Rules not processed for user `5a0fd07d66ae48a9b46f6a2ce9996326`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:37.138428 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-679515b9-c50c-411d-9bf5-10a83e50de1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:37.203960 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0631b2-948a-4e3a-95d1-95b3c08d74ec admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:37.206114 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-fc0631b2-948a-4e3a-95d1-95b3c08d74ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:37.400075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0631b2-948a-4e3a-95d1-95b3c08d74ec admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:37.416402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c41c5-eabf-441c-b29c-41ef289b18b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:37.417272 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c41c5-eabf-441c-b29c-41ef289b18b6 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:37.418748 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c41c5-eabf-441c-b29c-41ef289b18b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:37.420167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c41c5-eabf-441c-b29c-41ef289b18b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:37.449250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c41c5-eabf-441c-b29c-41ef289b18b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:37.451984 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c41c5-eabf-441c-b29c-41ef289b18b6 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:37.485915 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 864/1744] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 09:47:37.531675 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 881/1745] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:37.555447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c41c5-eabf-441c-b29c-41ef289b18b6 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:37.557842 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-009c41c5-eabf-441c-b29c-41ef289b18b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:37.695521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c41c5-eabf-441c-b29c-41ef289b18b6 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:37.704533 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e5a0b-0623-4486-88cd-f0a02ae863cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:37.705607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e5a0b-0623-4486-88cd-f0a02ae863cc service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:37.706086 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e5a0b-0623-4486-88cd-f0a02ae863cc service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:37.706567 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e5a0b-0623-4486-88cd-f0a02ae863cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:37.719501 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e5a0b-0623-4486-88cd-f0a02ae863cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:37.721113 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e5a0b-0623-4486-88cd-f0a02ae863cc service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:37.739486 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 865/1746] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:37.772449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679515b9-c50c-411d-9bf5-10a83e50de1d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:37.922413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679515b9-c50c-411d-9bf5-10a83e50de1d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:37.939942 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0a102-778e-4a51-8e97-7556a73f36e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:37.941144 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0a102-778e-4a51-8e97-7556a73f36e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:37.941578 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0a102-778e-4a51-8e97-7556a73f36e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:37.941998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0a102-778e-4a51-8e97-7556a73f36e1 admin admin] PATH_INFO: `/v3/users/8aa346d326624c28bd91b13d946a1afd` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:37.943406 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec0a102-778e-4a51-8e97-7556a73f36e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8aa346d326624c28bd91b13d946a1afd)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:37.964854 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec0a102-778e-4a51-8e97-7556a73f36e1 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:38.078286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-fec0a102-778e-4a51-8e97-7556a73f36e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8aa346d326624c28bd91b13d946a1afd'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:38.106693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-fec0a102-778e-4a51-8e97-7556a73f36e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8aa346d326624c28bd91b13d946a1afd'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:38.108767 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-fec0a102-778e-4a51-8e97-7556a73f36e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8aa346d326624c28bd91b13d946a1afd'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:38.126990 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-fec0a102-778e-4a51-8e97-7556a73f36e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8aa346d326624c28bd91b13d946a1afd'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:38.127746 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-fec0a102-778e-4a51-8e97-7556a73f36e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8aa346d326624c28bd91b13d946a1afd'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:38.193748 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 882/1747] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:47:37 2022] DELETE /identity/v3/users/8aa346d326624c28bd91b13d946a1afd => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:38.198505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2001a44-bb2a-4e97-b1ae-317ec26e31fc None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:38.198965 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2001a44-bb2a-4e97-b1ae-317ec26e31fc None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:38.199058 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2001a44-bb2a-4e97-b1ae-317ec26e31fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:38.254395 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-f2001a44-bb2a-4e97-b1ae-317ec26e31fc None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:38.255867 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f2001a44-bb2a-4e97-b1ae-317ec26e31fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:38.384926 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 866/1748] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:38.422718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0a102-778e-4a51-8e97-7556a73f36e1 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:38.425833 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-fec0a102-778e-4a51-8e97-7556a73f36e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:38.635016 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0a102-778e-4a51-8e97-7556a73f36e1 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:38.641320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4cf5e-0b7f-4d1b-b4ed-3c750675b33e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:38.643032 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4cf5e-0b7f-4d1b-b4ed-3c750675b33e service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:38.643478 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4cf5e-0b7f-4d1b-b4ed-3c750675b33e service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:38.654704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4cf5e-0b7f-4d1b-b4ed-3c750675b33e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:38.685007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4cf5e-0b7f-4d1b-b4ed-3c750675b33e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:38.687806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4cf5e-0b7f-4d1b-b4ed-3c750675b33e service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:38.704084 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 883/1749] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:39.366833 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe06f9-56af-4ba1-8391-5309a151a5d5 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:39.367386 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe06f9-56af-4ba1-8391-5309a151a5d5 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:39.367386 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe06f9-56af-4ba1-8391-5309a151a5d5 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:39.370313 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 867/1750] 10.222.0.26 () {54 vars in 921 bytes} [Mon May 9 09:47:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:47:39.394488 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1b7cd-f42e-42ea-91bc-f4f686fbe3dd None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:39.395045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1b7cd-f42e-42ea-91bc-f4f686fbe3dd None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:39.395045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1b7cd-f42e-42ea-91bc-f4f686fbe3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:39.436879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-b3b1b7cd-f42e-42ea-91bc-f4f686fbe3dd None None] MFA Rules not processed for user `6362303cb918486c8525225914332ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:39.438176 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b3b1b7cd-f42e-42ea-91bc-f4f686fbe3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:39.677363 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 884/1751] 10.222.0.26 () {58 vars in 1018 bytes} [Mon May 9 09:47:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:39.717520 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe06f9-56af-4ba1-8391-5309a151a5d5 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:39.740974 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-bfbe06f9-56af-4ba1-8391-5309a151a5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:39.763398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1b7cd-f42e-42ea-91bc-f4f686fbe3dd None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:39.765121 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b3b1b7cd-f42e-42ea-91bc-f4f686fbe3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:39.791784 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1b7cd-f42e-42ea-91bc-f4f686fbe3dd None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:39.798351 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0d930-c9e7-40b9-b538-0fad1a1df491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:39.818180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0d930-c9e7-40b9-b538-0fad1a1df491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:39.824865 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0d930-c9e7-40b9-b538-0fad1a1df491 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:39.825387 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0d930-c9e7-40b9-b538-0fad1a1df491 admin admin] PATH_INFO: `/v3/users/7e95fce19921454a99e6071423db7932` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:39.828351 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e0d930-c9e7-40b9-b538-0fad1a1df491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e95fce19921454a99e6071423db7932)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:39.832230 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e0d930-c9e7-40b9-b538-0fad1a1df491 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:39.842656 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe06f9-56af-4ba1-8391-5309a151a5d5 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:39.860716 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08700f-9ae9-4b6c-af7d-8dffdf84bcdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:39.865971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08700f-9ae9-4b6c-af7d-8dffdf84bcdf service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:39.865971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08700f-9ae9-4b6c-af7d-8dffdf84bcdf service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:39.865971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08700f-9ae9-4b6c-af7d-8dffdf84bcdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:39.876220 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc08700f-9ae9-4b6c-af7d-8dffdf84bcdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:39.878261 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc08700f-9ae9-4b6c-af7d-8dffdf84bcdf service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:39.928909 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 868/1752] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:39.932645 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08700f-9ae9-4b6c-af7d-8dffdf84bcdf service nova] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:39.940479 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-dc08700f-9ae9-4b6c-af7d-8dffdf84bcdf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:39.990179 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08700f-9ae9-4b6c-af7d-8dffdf84bcdf service nova] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:39.992413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-34e0d930-c9e7-40b9-b538-0fad1a1df491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e95fce19921454a99e6071423db7932'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:40.012290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e100ef2-aab4-4ee0-95b3-3b227419fa24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:40.014704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e100ef2-aab4-4ee0-95b3-3b227419fa24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:40.015373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e100ef2-aab4-4ee0-95b3-3b227419fa24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:40.015997 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e100ef2-aab4-4ee0-95b3-3b227419fa24 admin admin] PATH_INFO: `/v3/projects/1387607ad9e14443bede2a41a717315f` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:40.017394 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-34e0d930-c9e7-40b9-b538-0fad1a1df491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e95fce19921454a99e6071423db7932'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:40.018176 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-34e0d930-c9e7-40b9-b538-0fad1a1df491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e95fce19921454a99e6071423db7932'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:40.018787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e100ef2-aab4-4ee0-95b3-3b227419fa24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1387607ad9e14443bede2a41a717315f)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:40.022933 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e100ef2-aab4-4ee0-95b3-3b227419fa24 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:40.031413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-34e0d930-c9e7-40b9-b538-0fad1a1df491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e95fce19921454a99e6071423db7932'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:40.032457 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-34e0d930-c9e7-40b9-b538-0fad1a1df491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e95fce19921454a99e6071423db7932'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:40.068888 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 885/1753] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:47:39 2022] DELETE /identity/v3/users/7e95fce19921454a99e6071423db7932 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:40.081805 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80815f21-f68a-4c9d-80c3-8db66a03bf4b None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:40.083543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80815f21-f68a-4c9d-80c3-8db66a03bf4b None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:40.083966 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80815f21-f68a-4c9d-80c3-8db66a03bf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:40.135345 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-80815f21-f68a-4c9d-80c3-8db66a03bf4b None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:40.136693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-80815f21-f68a-4c9d-80c3-8db66a03bf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:40.225508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-2e100ef2-aab4-4ee0-95b3-3b227419fa24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1387607ad9e14443bede2a41a717315f', 'request_id': 'req-2e100ef2-aab4-4ee0-95b3-3b227419fa24'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:40.240483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 886/1754] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:40.241652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80815f21-f68a-4c9d-80c3-8db66a03bf4b None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:40.243213 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-80815f21-f68a-4c9d-80c3-8db66a03bf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:40.277776 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-2e100ef2-aab4-4ee0-95b3-3b227419fa24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1387607ad9e14443bede2a41a717315f', 'request_id': 'req-2e100ef2-aab4-4ee0-95b3-3b227419fa24'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:40.297846 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-2e100ef2-aab4-4ee0-95b3-3b227419fa24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1387607ad9e14443bede2a41a717315f', 'request_id': 'req-2e100ef2-aab4-4ee0-95b3-3b227419fa24'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:40.313209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80815f21-f68a-4c9d-80c3-8db66a03bf4b None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:40.349952 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-2e100ef2-aab4-4ee0-95b3-3b227419fa24 admin admin] The token cache is being invalidate because project 1387607ad9e14443bede2a41a717315f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:47:40.350872 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-2e100ef2-aab4-4ee0-95b3-3b227419fa24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:40.353245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-2e100ef2-aab4-4ee0-95b3-3b227419fa24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:40.362049 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 869/1755] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:47:39 2022] DELETE /identity/v3/projects/1387607ad9e14443bede2a41a717315f => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:40.365631 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e100ef2-aab4-4ee0-95b3-3b227419fa24 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:40.367686 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-2e100ef2-aab4-4ee0-95b3-3b227419fa24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:40.426345 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced5b7f-d9db-443d-a47e-a8b658b22d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:40.427290 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced5b7f-d9db-443d-a47e-a8b658b22d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:40.427633 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced5b7f-d9db-443d-a47e-a8b658b22d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:40.427910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced5b7f-d9db-443d-a47e-a8b658b22d48 admin admin] PATH_INFO: `/v3/users/9f2d5d83d69f4e6f8df4a8192aa30880` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:40.437623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced5b7f-d9db-443d-a47e-a8b658b22d48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f2d5d83d69f4e6f8df4a8192aa30880)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:40.439611 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced5b7f-d9db-443d-a47e-a8b658b22d48 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:40.446967 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e100ef2-aab4-4ee0-95b3-3b227419fa24 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:40.451060 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e25347-a6bd-4160-8608-a9f6da968401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:40.451794 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e25347-a6bd-4160-8608-a9f6da968401 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:40.452241 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e25347-a6bd-4160-8608-a9f6da968401 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:40.452604 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e25347-a6bd-4160-8608-a9f6da968401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:40.455873 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-c7e25347-a6bd-4160-8608-a9f6da968401 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:40.473549 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e25347-a6bd-4160-8608-a9f6da968401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:40.476010 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e25347-a6bd-4160-8608-a9f6da968401 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:40.494678 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 870/1756] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:40.495856 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e25347-a6bd-4160-8608-a9f6da968401 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:40.497752 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-c7e25347-a6bd-4160-8608-a9f6da968401 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:40.510602 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e25347-a6bd-4160-8608-a9f6da968401 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:40.517999 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e2a0a-0662-4269-8f20-caa65a834e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:40.519127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e2a0a-0662-4269-8f20-caa65a834e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:40.519554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e2a0a-0662-4269-8f20-caa65a834e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:40.519970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e2a0a-0662-4269-8f20-caa65a834e1b admin admin] PATH_INFO: `/v3/users/5a0fd07d66ae48a9b46f6a2ce9996326` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:40.521564 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e2a0a-0662-4269-8f20-caa65a834e1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a0fd07d66ae48a9b46f6a2ce9996326)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:40.524050 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e2a0a-0662-4269-8f20-caa65a834e1b admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:40.525046 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8ced5b7f-d9db-443d-a47e-a8b658b22d48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f2d5d83d69f4e6f8df4a8192aa30880'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:40.541805 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8ced5b7f-d9db-443d-a47e-a8b658b22d48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f2d5d83d69f4e6f8df4a8192aa30880'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:40.543187 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8ced5b7f-d9db-443d-a47e-a8b658b22d48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f2d5d83d69f4e6f8df4a8192aa30880'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:40.564686 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8ced5b7f-d9db-443d-a47e-a8b658b22d48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f2d5d83d69f4e6f8df4a8192aa30880'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:40.565451 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8ced5b7f-d9db-443d-a47e-a8b658b22d48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f2d5d83d69f4e6f8df4a8192aa30880'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:40.585218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 887/1757] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:47:40 2022] DELETE /identity/v3/users/9f2d5d83d69f4e6f8df4a8192aa30880 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:40.589966 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb67a0-e13d-43ae-8946-0bd95e410f21 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:40.590325 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb67a0-e13d-43ae-8946-0bd95e410f21 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:40.590677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb67a0-e13d-43ae-8946-0bd95e410f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:40.592304 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-775e2a0a-0662-4269-8f20-caa65a834e1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a0fd07d66ae48a9b46f6a2ce9996326'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:40.601642 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-775e2a0a-0662-4269-8f20-caa65a834e1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a0fd07d66ae48a9b46f6a2ce9996326'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:40.609396 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-e6bb67a0-e13d-43ae-8946-0bd95e410f21 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:40.610772 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e6bb67a0-e13d-43ae-8946-0bd95e410f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:40.611393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-775e2a0a-0662-4269-8f20-caa65a834e1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a0fd07d66ae48a9b46f6a2ce9996326'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:40.612035 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-775e2a0a-0662-4269-8f20-caa65a834e1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a0fd07d66ae48a9b46f6a2ce9996326'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:40.612625 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-775e2a0a-0662-4269-8f20-caa65a834e1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a0fd07d66ae48a9b46f6a2ce9996326'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:40.636198 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 871/1758] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:47:40 2022] DELETE /identity/v3/users/5a0fd07d66ae48a9b46f6a2ce9996326 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:40.678720 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 888/1759] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:40.696372 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e2a0a-0662-4269-8f20-caa65a834e1b admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:40.698232 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-775e2a0a-0662-4269-8f20-caa65a834e1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:40.757811 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e2a0a-0662-4269-8f20-caa65a834e1b admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:40.761213 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366574b5-bab8-46ed-a340-44e4b328e64c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:40.761855 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366574b5-bab8-46ed-a340-44e4b328e64c service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:40.762128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366574b5-bab8-46ed-a340-44e4b328e64c service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:40.762393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366574b5-bab8-46ed-a340-44e4b328e64c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:40.822490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366574b5-bab8-46ed-a340-44e4b328e64c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:40.823939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366574b5-bab8-46ed-a340-44e4b328e64c service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:40.833609 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 872/1760] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:40.925785 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb67a0-e13d-43ae-8946-0bd95e410f21 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:40.932793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e6bb67a0-e13d-43ae-8946-0bd95e410f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:40.957241 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb67a0-e13d-43ae-8946-0bd95e410f21 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:40.963943 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f823df-37e8-48b7-8286-d0fdda4eddf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:40.966040 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f823df-37e8-48b7-8286-d0fdda4eddf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:40.966512 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f823df-37e8-48b7-8286-d0fdda4eddf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:40.966919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f823df-37e8-48b7-8286-d0fdda4eddf0 admin admin] PATH_INFO: `/v3/projects/23b8170c316f43eaa233c612251d499a` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:40.969925 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f823df-37e8-48b7-8286-d0fdda4eddf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23b8170c316f43eaa233c612251d499a)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:40.972956 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f823df-37e8-48b7-8286-d0fdda4eddf0 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:41.049543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-a7f823df-37e8-48b7-8286-d0fdda4eddf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23b8170c316f43eaa233c612251d499a', 'request_id': 'req-a7f823df-37e8-48b7-8286-d0fdda4eddf0'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.059348 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-a7f823df-37e8-48b7-8286-d0fdda4eddf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23b8170c316f43eaa233c612251d499a', 'request_id': 'req-a7f823df-37e8-48b7-8286-d0fdda4eddf0'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.066897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-a7f823df-37e8-48b7-8286-d0fdda4eddf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23b8170c316f43eaa233c612251d499a', 'request_id': 'req-a7f823df-37e8-48b7-8286-d0fdda4eddf0'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.073059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366574b5-bab8-46ed-a340-44e4b328e64c service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:41.075175 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-366574b5-bab8-46ed-a340-44e4b328e64c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:41.090427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366574b5-bab8-46ed-a340-44e4b328e64c service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:41.101580 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-a7f823df-37e8-48b7-8286-d0fdda4eddf0 admin admin] The token cache is being invalidate because project 23b8170c316f43eaa233c612251d499a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:47:41.102793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-a7f823df-37e8-48b7-8286-d0fdda4eddf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.104100 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-a7f823df-37e8-48b7-8286-d0fdda4eddf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.113675 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 889/1761] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:47:40 2022] DELETE /identity/v3/projects/23b8170c316f43eaa233c612251d499a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:41.123110 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f823df-37e8-48b7-8286-d0fdda4eddf0 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:41.130866 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-a7f823df-37e8-48b7-8286-d0fdda4eddf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:41.181957 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1708bf-ead8-4aaa-a77c-385b281ae1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:41.182934 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1708bf-ead8-4aaa-a77c-385b281ae1ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:41.183230 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1708bf-ead8-4aaa-a77c-385b281ae1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:41.183498 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1708bf-ead8-4aaa-a77c-385b281ae1ad admin admin] PATH_INFO: `/v3/projects/3ea9ffde7af44e6a9e1b6c5c79232eb0` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:41.185242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1708bf-ead8-4aaa-a77c-385b281ae1ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ea9ffde7af44e6a9e1b6c5c79232eb0)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:41.187275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1708bf-ead8-4aaa-a77c-385b281ae1ad admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:41.201229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f823df-37e8-48b7-8286-d0fdda4eddf0 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:41.205989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0cf21-454f-470d-99a8-4b13c79a8c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:41.206863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0cf21-454f-470d-99a8-4b13c79a8c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:41.207138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0cf21-454f-470d-99a8-4b13c79a8c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:41.207405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0cf21-454f-470d-99a8-4b13c79a8c38 admin admin] PATH_INFO: `/v3/users/843391e07bc0400eb41661ea0b2b1963` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:41.208593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0cf21-454f-470d-99a8-4b13c79a8c38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=843391e07bc0400eb41661ea0b2b1963)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:41.210538 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0cf21-454f-470d-99a8-4b13c79a8c38 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:41.235645 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-db1708bf-ead8-4aaa-a77c-385b281ae1ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ea9ffde7af44e6a9e1b6c5c79232eb0', 'request_id': 'req-db1708bf-ead8-4aaa-a77c-385b281ae1ad'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.242331 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-db1708bf-ead8-4aaa-a77c-385b281ae1ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ea9ffde7af44e6a9e1b6c5c79232eb0', 'request_id': 'req-db1708bf-ead8-4aaa-a77c-385b281ae1ad'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.247769 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-db1708bf-ead8-4aaa-a77c-385b281ae1ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ea9ffde7af44e6a9e1b6c5c79232eb0', 'request_id': 'req-db1708bf-ead8-4aaa-a77c-385b281ae1ad'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.260977 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-db1708bf-ead8-4aaa-a77c-385b281ae1ad admin admin] The token cache is being invalidate because project 3ea9ffde7af44e6a9e1b6c5c79232eb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:47:41.261207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-db1708bf-ead8-4aaa-a77c-385b281ae1ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.261895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-db1708bf-ead8-4aaa-a77c-385b281ae1ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.262914 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-64d0cf21-454f-470d-99a8-4b13c79a8c38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '843391e07bc0400eb41661ea0b2b1963'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.266694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 873/1762] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:47:41 2022] DELETE /identity/v3/projects/3ea9ffde7af44e6a9e1b6c5c79232eb0 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:47:41.266694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1708bf-ead8-4aaa-a77c-385b281ae1ad admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:41.266694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-db1708bf-ead8-4aaa-a77c-385b281ae1ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:41.278963 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-64d0cf21-454f-470d-99a8-4b13c79a8c38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '843391e07bc0400eb41661ea0b2b1963'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.279713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-64d0cf21-454f-470d-99a8-4b13c79a8c38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '843391e07bc0400eb41661ea0b2b1963'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.288111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-64d0cf21-454f-470d-99a8-4b13c79a8c38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '843391e07bc0400eb41661ea0b2b1963'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.288858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-64d0cf21-454f-470d-99a8-4b13c79a8c38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '843391e07bc0400eb41661ea0b2b1963'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.307347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 890/1763] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:47:41 2022] DELETE /identity/v3/users/843391e07bc0400eb41661ea0b2b1963 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:41.309173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bdabe-9c51-410e-beaa-000176d98f75 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:41.309463 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bdabe-9c51-410e-beaa-000176d98f75 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:41.309739 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bdabe-9c51-410e-beaa-000176d98f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:41.330314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-ae3bdabe-9c51-410e-beaa-000176d98f75 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:41.331352 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-ae3bdabe-9c51-410e-beaa-000176d98f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:41.339429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1708bf-ead8-4aaa-a77c-385b281ae1ad admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:41.397913 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 891/1764] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:41.406769 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2948b8-9a6b-44a8-a616-0df06d9b37d9 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:41.407095 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2948b8-9a6b-44a8-a616-0df06d9b37d9 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:41.407363 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2948b8-9a6b-44a8-a616-0df06d9b37d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:41.412359 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44ba3e-021a-4bbe-abdc-2ea30fa3ce42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:41.413892 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44ba3e-021a-4bbe-abdc-2ea30fa3ce42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:41.414368 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44ba3e-021a-4bbe-abdc-2ea30fa3ce42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:41.414639 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44ba3e-021a-4bbe-abdc-2ea30fa3ce42 admin admin] PATH_INFO: `/v3/projects/005420558a4a41ec904e88728581fd5e` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:41.416803 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44ba3e-021a-4bbe-abdc-2ea30fa3ce42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=005420558a4a41ec904e88728581fd5e)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:41.418916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44ba3e-021a-4bbe-abdc-2ea30fa3ce42 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:41.427114 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-9a2948b8-9a6b-44a8-a616-0df06d9b37d9 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:41.428368 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-9a2948b8-9a6b-44a8-a616-0df06d9b37d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:41.437769 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 892/1765] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:41.446616 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2948b8-9a6b-44a8-a616-0df06d9b37d9 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:41.456947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2948b8-9a6b-44a8-a616-0df06d9b37d9 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:41.461893 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8878755-61e8-4f01-8c16-54a78f31ae9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:41.462687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8878755-61e8-4f01-8c16-54a78f31ae9a admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:41.463013 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8878755-61e8-4f01-8c16-54a78f31ae9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:41.463234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8878755-61e8-4f01-8c16-54a78f31ae9a admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:41.464199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8878755-61e8-4f01-8c16-54a78f31ae9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:47:41.464540 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8878755-61e8-4f01-8c16-54a78f31ae9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:41.466363 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8878755-61e8-4f01-8c16-54a78f31ae9a admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:41.477898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-1c44ba3e-021a-4bbe-abdc-2ea30fa3ce42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '005420558a4a41ec904e88728581fd5e', 'request_id': 'req-1c44ba3e-021a-4bbe-abdc-2ea30fa3ce42'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.482788 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 893/1766] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:47:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:47:41.483813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8878755-61e8-4f01-8c16-54a78f31ae9a admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:41.485387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-1c44ba3e-021a-4bbe-abdc-2ea30fa3ce42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '005420558a4a41ec904e88728581fd5e', 'request_id': 'req-1c44ba3e-021a-4bbe-abdc-2ea30fa3ce42'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.491097 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-1c44ba3e-021a-4bbe-abdc-2ea30fa3ce42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '005420558a4a41ec904e88728581fd5e', 'request_id': 'req-1c44ba3e-021a-4bbe-abdc-2ea30fa3ce42'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.494425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8878755-61e8-4f01-8c16-54a78f31ae9a admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:41.508542 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-1c44ba3e-021a-4bbe-abdc-2ea30fa3ce42 admin admin] The token cache is being invalidate because project 005420558a4a41ec904e88728581fd5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:47:41.508870 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-1c44ba3e-021a-4bbe-abdc-2ea30fa3ce42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.509521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-1c44ba3e-021a-4bbe-abdc-2ea30fa3ce42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.511418 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 874/1767] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:47:41 2022] DELETE /identity/v3/projects/005420558a4a41ec904e88728581fd5e => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:41.515115 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44ba3e-021a-4bbe-abdc-2ea30fa3ce42 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:41.516480 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-1c44ba3e-021a-4bbe-abdc-2ea30fa3ce42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:41.568106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e0f95-cb79-42e3-bf17-0f4f3d354440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:41.568812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e0f95-cb79-42e3-bf17-0f4f3d354440 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:41.569093 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e0f95-cb79-42e3-bf17-0f4f3d354440 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:41.569363 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e0f95-cb79-42e3-bf17-0f4f3d354440 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:41.570058 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e0f95-cb79-42e3-bf17-0f4f3d354440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:41.571973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e0f95-cb79-42e3-bf17-0f4f3d354440 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:41.581832 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44ba3e-021a-4bbe-abdc-2ea30fa3ce42 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:41.587366 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e77ca-63b8-453c-914d-a3a07877e5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:41.588352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e77ca-63b8-453c-914d-a3a07877e5c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:41.588835 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e77ca-63b8-453c-914d-a3a07877e5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:41.589274 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e77ca-63b8-453c-914d-a3a07877e5c8 admin admin] PATH_INFO: `/v3/users/41599522fd3a4beab6f709858c204e5b` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:41.590858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829e77ca-63b8-453c-914d-a3a07877e5c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41599522fd3a4beab6f709858c204e5b)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:41.593137 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829e77ca-63b8-453c-914d-a3a07877e5c8 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:41.611505 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 894/1768] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:41 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:41.615278 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e0f95-cb79-42e3-bf17-0f4f3d354440 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:41.617315 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-f09e0f95-cb79-42e3-bf17-0f4f3d354440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:41.655185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-829e77ca-63b8-453c-914d-a3a07877e5c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41599522fd3a4beab6f709858c204e5b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.663598 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-829e77ca-63b8-453c-914d-a3a07877e5c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41599522fd3a4beab6f709858c204e5b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.673506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-829e77ca-63b8-453c-914d-a3a07877e5c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41599522fd3a4beab6f709858c204e5b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.674522 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-829e77ca-63b8-453c-914d-a3a07877e5c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41599522fd3a4beab6f709858c204e5b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.675754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-829e77ca-63b8-453c-914d-a3a07877e5c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41599522fd3a4beab6f709858c204e5b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.701407 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 875/1769] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:47:41 2022] DELETE /identity/v3/users/41599522fd3a4beab6f709858c204e5b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:47:41.711766 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e0f95-cb79-42e3-bf17-0f4f3d354440 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:41.744016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e77ca-63b8-453c-914d-a3a07877e5c8 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:41.745706 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-829e77ca-63b8-453c-914d-a3a07877e5c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:41.782335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6e59f-b4ab-4c36-be5a-e920c381cf89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:41.783090 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6e59f-b4ab-4c36-be5a-e920c381cf89 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:41.783671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6e59f-b4ab-4c36-be5a-e920c381cf89 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:41.783671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6e59f-b4ab-4c36-be5a-e920c381cf89 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:41.784479 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6e59f-b4ab-4c36-be5a-e920c381cf89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:41.786753 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6e59f-b4ab-4c36-be5a-e920c381cf89 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:41.814362 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 895/1770] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:41 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:47:41.817631 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6e59f-b4ab-4c36-be5a-e920c381cf89 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:41.819617 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-a3c6e59f-b4ab-4c36-be5a-e920c381cf89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:41.827712 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e77ca-63b8-453c-914d-a3a07877e5c8 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:41.832375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c60fb-2d97-48e8-832e-700cb810a338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:41.833312 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c60fb-2d97-48e8-832e-700cb810a338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:41.833511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c60fb-2d97-48e8-832e-700cb810a338 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:41.833787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c60fb-2d97-48e8-832e-700cb810a338 admin admin] PATH_INFO: `/v3/projects/b5b4118142524da281c8ba137e833456` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:41.834988 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c60fb-2d97-48e8-832e-700cb810a338 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5b4118142524da281c8ba137e833456)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:41.835455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6e59f-b4ab-4c36-be5a-e920c381cf89 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:41.836944 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c60fb-2d97-48e8-832e-700cb810a338 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:41.841092 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32235963-0338-4cb0-86ed-b3c315c06243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:41.842698 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32235963-0338-4cb0-86ed-b3c315c06243 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:41.844516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32235963-0338-4cb0-86ed-b3c315c06243 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:41.845024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32235963-0338-4cb0-86ed-b3c315c06243 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:41.845994 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32235963-0338-4cb0-86ed-b3c315c06243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:41.847781 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32235963-0338-4cb0-86ed-b3c315c06243 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:41.864807 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 896/1771] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:41.866249 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32235963-0338-4cb0-86ed-b3c315c06243 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:41.877150 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32235963-0338-4cb0-86ed-b3c315c06243 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:41.884980 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591cefea-6f24-4be4-9399-ffb1cbdba1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:41.885665 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591cefea-6f24-4be4-9399-ffb1cbdba1db admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:41.885948 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591cefea-6f24-4be4-9399-ffb1cbdba1db admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:41.886267 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591cefea-6f24-4be4-9399-ffb1cbdba1db admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:41.886948 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591cefea-6f24-4be4-9399-ffb1cbdba1db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:41.898280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591cefea-6f24-4be4-9399-ffb1cbdba1db admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:41.921073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 897/1772] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:41 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:41.925049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591cefea-6f24-4be4-9399-ffb1cbdba1db admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:41.933812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-ac7c60fb-2d97-48e8-832e-700cb810a338 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5b4118142524da281c8ba137e833456', 'request_id': 'req-ac7c60fb-2d97-48e8-832e-700cb810a338'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.942523 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-ac7c60fb-2d97-48e8-832e-700cb810a338 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5b4118142524da281c8ba137e833456', 'request_id': 'req-ac7c60fb-2d97-48e8-832e-700cb810a338'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.944552 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591cefea-6f24-4be4-9399-ffb1cbdba1db admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:41.947964 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-ac7c60fb-2d97-48e8-832e-700cb810a338 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5b4118142524da281c8ba137e833456', 'request_id': 'req-ac7c60fb-2d97-48e8-832e-700cb810a338'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.969152 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-ac7c60fb-2d97-48e8-832e-700cb810a338 admin admin] The token cache is being invalidate because project b5b4118142524da281c8ba137e833456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:47:41.970274 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-ac7c60fb-2d97-48e8-832e-700cb810a338 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.971313 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-ac7c60fb-2d97-48e8-832e-700cb810a338 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:41.973411 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 876/1773] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:47:41 2022] DELETE /identity/v3/projects/b5b4118142524da281c8ba137e833456 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:47:41.978457 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac04d8-4099-43f7-b0f2-ab4802157a7b None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:41.978813 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac04d8-4099-43f7-b0f2-ab4802157a7b None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:41.979956 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac04d8-4099-43f7-b0f2-ab4802157a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:42.005849 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-d4ac04d8-4099-43f7-b0f2-ab4802157a7b None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:42.007205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-d4ac04d8-4099-43f7-b0f2-ab4802157a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:42.009784 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda60027-5528-4ec0-8384-4bc6f8f5fcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:42.010860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda60027-5528-4ec0-8384-4bc6f8f5fcd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:42.011251 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda60027-5528-4ec0-8384-4bc6f8f5fcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:42.011593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda60027-5528-4ec0-8384-4bc6f8f5fcd7 admin admin] PATH_INFO: `/v3/projects/fc1d9b49a83f4b30abb84e6727f44d76/users/c54d3b25afc74c13a9f6b85ab567fe0a/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:42.016568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 877/1774] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:47:42.024548 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af87ec6-8d14-4878-8b50-1b0ebf0e727a None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:42.024859 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af87ec6-8d14-4878-8b50-1b0ebf0e727a None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:42.025755 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af87ec6-8d14-4878-8b50-1b0ebf0e727a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:42.032751 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda60027-5528-4ec0-8384-4bc6f8f5fcd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc1d9b49a83f4b30abb84e6727f44d76, user_id=c54d3b25afc74c13a9f6b85ab567fe0a, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:42.035364 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda60027-5528-4ec0-8384-4bc6f8f5fcd7 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:42.062291 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 898/1775] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:41 2022] PUT /identity/v3/projects/fc1d9b49a83f4b30abb84e6727f44d76/users/c54d3b25afc74c13a9f6b85ab567fe0a/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:42.062291 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda60027-5528-4ec0-8384-4bc6f8f5fcd7 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:42.064868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-cda60027-5528-4ec0-8384-4bc6f8f5fcd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:42.067271 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-3af87ec6-8d14-4878-8b50-1b0ebf0e727a None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:42.068502 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-3af87ec6-8d14-4878-8b50-1b0ebf0e727a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:42.154770 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda60027-5528-4ec0-8384-4bc6f8f5fcd7 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:42.160071 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4fbbf-c637-4c1f-a4d8-bfcf6945a150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:42.160999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4fbbf-c637-4c1f-a4d8-bfcf6945a150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:42.161271 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4fbbf-c637-4c1f-a4d8-bfcf6945a150 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:42.161536 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4fbbf-c637-4c1f-a4d8-bfcf6945a150 admin admin] PATH_INFO: `/v3/projects/da3442de7b7644d7a34e5348cc3ced10` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:42.164458 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4fbbf-c637-4c1f-a4d8-bfcf6945a150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da3442de7b7644d7a34e5348cc3ced10)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:42.166608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4fbbf-c637-4c1f-a4d8-bfcf6945a150 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:42.170046 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 878/1776] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:47:42.171527 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfcc64-2bb7-468c-b425-abf419afe18d None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:42.171809 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfcc64-2bb7-468c-b425-abf419afe18d None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:42.172079 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfcc64-2bb7-468c-b425-abf419afe18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:42.220560 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-95dfcc64-2bb7-468c-b425-abf419afe18d None None] MFA Rules not processed for user `c54d3b25afc74c13a9f6b85ab567fe0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:42.225522 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-95dfcc64-2bb7-468c-b425-abf419afe18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:42.239898 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-f4e4fbbf-c637-4c1f-a4d8-bfcf6945a150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da3442de7b7644d7a34e5348cc3ced10', 'request_id': 'req-f4e4fbbf-c637-4c1f-a4d8-bfcf6945a150'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:42.246743 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-f4e4fbbf-c637-4c1f-a4d8-bfcf6945a150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da3442de7b7644d7a34e5348cc3ced10', 'request_id': 'req-f4e4fbbf-c637-4c1f-a4d8-bfcf6945a150'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:42.252620 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-f4e4fbbf-c637-4c1f-a4d8-bfcf6945a150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da3442de7b7644d7a34e5348cc3ced10', 'request_id': 'req-f4e4fbbf-c637-4c1f-a4d8-bfcf6945a150'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:42.268693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-f4e4fbbf-c637-4c1f-a4d8-bfcf6945a150 admin admin] The token cache is being invalidate because project da3442de7b7644d7a34e5348cc3ced10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:47:42.268994 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-f4e4fbbf-c637-4c1f-a4d8-bfcf6945a150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:42.269651 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-f4e4fbbf-c637-4c1f-a4d8-bfcf6945a150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:42.280805 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 899/1777] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:47:42 2022] DELETE /identity/v3/projects/da3442de7b7644d7a34e5348cc3ced10 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:42.282889 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4fbbf-c637-4c1f-a4d8-bfcf6945a150 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:42.284511 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-f4e4fbbf-c637-4c1f-a4d8-bfcf6945a150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:42.308762 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 879/1778] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:42.310442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f98ead-ada5-4a8a-8886-e6c926f16eab None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:42.310744 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f98ead-ada5-4a8a-8886-e6c926f16eab None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:42.311274 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f98ead-ada5-4a8a-8886-e6c926f16eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:42.341981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-d9f98ead-ada5-4a8a-8886-e6c926f16eab None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:42.343267 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-d9f98ead-ada5-4a8a-8886-e6c926f16eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:42.425565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4fbbf-c637-4c1f-a4d8-bfcf6945a150 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:42.437060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568bf30-b9ba-4be6-a14f-ca95ff3d223d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:42.437743 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568bf30-b9ba-4be6-a14f-ca95ff3d223d admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:42.438019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568bf30-b9ba-4be6-a14f-ca95ff3d223d admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:42.438394 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568bf30-b9ba-4be6-a14f-ca95ff3d223d admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:42.439251 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568bf30-b9ba-4be6-a14f-ca95ff3d223d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:47:42.439571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568bf30-b9ba-4be6-a14f-ca95ff3d223d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:42.441645 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568bf30-b9ba-4be6-a14f-ca95ff3d223d admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:42.445967 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 880/1779] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:42.447394 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f98ead-ada5-4a8a-8886-e6c926f16eab None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:42.448965 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-d9f98ead-ada5-4a8a-8886-e6c926f16eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:42.461323 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 900/1780] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:47:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:47:42.462905 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60ff27-4e0c-41b3-9fa8-598f6d4d70c8 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:42.463354 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60ff27-4e0c-41b3-9fa8-598f6d4d70c8 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:42.463806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60ff27-4e0c-41b3-9fa8-598f6d4d70c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:42.484649 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-eb60ff27-4e0c-41b3-9fa8-598f6d4d70c8 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:42.485809 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f98ead-ada5-4a8a-8886-e6c926f16eab None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:42.486662 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-eb60ff27-4e0c-41b3-9fa8-598f6d4d70c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:42.494123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11ed6a-72ad-4f4f-9e9a-846126bb84ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:42.494894 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11ed6a-72ad-4f4f-9e9a-846126bb84ca admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:42.495345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11ed6a-72ad-4f4f-9e9a-846126bb84ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:42.495898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11ed6a-72ad-4f4f-9e9a-846126bb84ca admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:42.496434 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 901/1781] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:42.497338 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11ed6a-72ad-4f4f-9e9a-846126bb84ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:42.498392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60ff27-4e0c-41b3-9fa8-598f6d4d70c8 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:42.500384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11ed6a-72ad-4f4f-9e9a-846126bb84ca admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:42.509354 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60ff27-4e0c-41b3-9fa8-598f6d4d70c8 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:42.515426 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f60c0-bc3e-495e-9bef-56ea09aeb406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:42.516097 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f60c0-bc3e-495e-9bef-56ea09aeb406 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:42.516370 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f60c0-bc3e-495e-9bef-56ea09aeb406 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:42.516640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f60c0-bc3e-495e-9bef-56ea09aeb406 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:42.517364 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f60c0-bc3e-495e-9bef-56ea09aeb406 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:42.519610 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f60c0-bc3e-495e-9bef-56ea09aeb406 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:42.533705 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 881/1782] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:42 2022] POST /identity/v3/projects => generated 381 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:42.538583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11ed6a-72ad-4f4f-9e9a-846126bb84ca admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:42.563465 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 902/1783] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:42 2022] POST /identity/v3/projects => generated 357 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:42.564489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f60c0-bc3e-495e-9bef-56ea09aeb406 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:42.622252 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11ed6a-72ad-4f4f-9e9a-846126bb84ca admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:42.652029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f60c0-bc3e-495e-9bef-56ea09aeb406 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:42.657466 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96a691-76a0-4919-b506-892ca69bf775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:42.658143 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96a691-76a0-4919-b506-892ca69bf775 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:42.658418 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96a691-76a0-4919-b506-892ca69bf775 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:42.658687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96a691-76a0-4919-b506-892ca69bf775 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:42.659444 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c96a691-76a0-4919-b506-892ca69bf775 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:42.661513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c96a691-76a0-4919-b506-892ca69bf775 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:42.684572 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ed24e-f7ef-4cd2-bc6e-aa9bee0dbe0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:42.685593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ed24e-f7ef-4cd2-bc6e-aa9bee0dbe0a admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:42.686077 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ed24e-f7ef-4cd2-bc6e-aa9bee0dbe0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:42.686623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ed24e-f7ef-4cd2-bc6e-aa9bee0dbe0a admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:42.687434 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ed24e-f7ef-4cd2-bc6e-aa9bee0dbe0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:47:42.687799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ed24e-f7ef-4cd2-bc6e-aa9bee0dbe0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:42.689636 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ed24e-f7ef-4cd2-bc6e-aa9bee0dbe0a admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:42.703818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 903/1784] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:42 2022] POST /identity/v3/users => generated 297 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:42.704954 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96a691-76a0-4919-b506-892ca69bf775 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:42.705455 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 882/1785] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:47:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:47:42.706632 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ed24e-f7ef-4cd2-bc6e-aa9bee0dbe0a admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:42.719544 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ed24e-f7ef-4cd2-bc6e-aa9bee0dbe0a admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:42.725111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96a691-76a0-4919-b506-892ca69bf775 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:42.725611 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309270d1-5bcf-4761-9c6f-d4f4c5ba59cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:42.726110 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309270d1-5bcf-4761-9c6f-d4f4c5ba59cc admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:42.726582 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309270d1-5bcf-4761-9c6f-d4f4c5ba59cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:42.726968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309270d1-5bcf-4761-9c6f-d4f4c5ba59cc admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:42.727759 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309270d1-5bcf-4761-9c6f-d4f4c5ba59cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:42.729465 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309270d1-5bcf-4761-9c6f-d4f4c5ba59cc admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:42.730365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50159cc9-30c4-4e50-9a53-a671efd19a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:42.731080 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50159cc9-30c4-4e50-9a53-a671efd19a83 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:42.731400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50159cc9-30c4-4e50-9a53-a671efd19a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:42.731673 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50159cc9-30c4-4e50-9a53-a671efd19a83 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:42.732358 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50159cc9-30c4-4e50-9a53-a671efd19a83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:42.734428 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50159cc9-30c4-4e50-9a53-a671efd19a83 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:42.744099 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 883/1786] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:42.744099 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309270d1-5bcf-4761-9c6f-d4f4c5ba59cc admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:42.755653 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309270d1-5bcf-4761-9c6f-d4f4c5ba59cc admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:42.760385 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1802b9-afd4-46c8-8a06-e39ca53ae5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:42.761148 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1802b9-afd4-46c8-8a06-e39ca53ae5e5 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:42.761482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1802b9-afd4-46c8-8a06-e39ca53ae5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:42.761816 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1802b9-afd4-46c8-8a06-e39ca53ae5e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:42.762529 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 904/1787] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:42 2022] POST /identity/v3/users => generated 279 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:42.762629 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1802b9-afd4-46c8-8a06-e39ca53ae5e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:42.764737 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1802b9-afd4-46c8-8a06-e39ca53ae5e5 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:42.766153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50159cc9-30c4-4e50-9a53-a671efd19a83 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:42.782232 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50159cc9-30c4-4e50-9a53-a671efd19a83 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:42.794542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add471a-c0e5-48c0-8acb-fb6df2ff8797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:42.794542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add471a-c0e5-48c0-8acb-fb6df2ff8797 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:42.794542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add471a-c0e5-48c0-8acb-fb6df2ff8797 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:42.794542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add471a-c0e5-48c0-8acb-fb6df2ff8797 admin admin] PATH_INFO: `/v3/projects/3357719665a946418154cd6596f4c71d/users/38a2a8c04b26416f97b2addba283fc68/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:42.799187 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 884/1788] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:42 2022] POST /identity/v3/projects => generated 401 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:42.799187 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1802b9-afd4-46c8-8a06-e39ca53ae5e5 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:42.807170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add471a-c0e5-48c0-8acb-fb6df2ff8797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3357719665a946418154cd6596f4c71d, user_id=38a2a8c04b26416f97b2addba283fc68, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:42.810167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add471a-c0e5-48c0-8acb-fb6df2ff8797 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:42.826084 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 905/1789] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:42 2022] PUT /identity/v3/projects/3357719665a946418154cd6596f4c71d/users/38a2a8c04b26416f97b2addba283fc68/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:42.826084 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add471a-c0e5-48c0-8acb-fb6df2ff8797 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:42.857388 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1802b9-afd4-46c8-8a06-e39ca53ae5e5 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:42.883575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add471a-c0e5-48c0-8acb-fb6df2ff8797 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:42.888399 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf76a5-0590-4d15-a9d1-966e1c43a4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:42.889094 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf76a5-0590-4d15-a9d1-966e1c43a4e4 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:42.889373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf76a5-0590-4d15-a9d1-966e1c43a4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:42.889644 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf76a5-0590-4d15-a9d1-966e1c43a4e4 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:42.890443 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf76a5-0590-4d15-a9d1-966e1c43a4e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:42.892371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf76a5-0590-4d15-a9d1-966e1c43a4e4 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:42.911501 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27227285-ab06-4b2d-a9c4-33c90b7f2070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:42.912170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27227285-ab06-4b2d-a9c4-33c90b7f2070 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:42.912447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27227285-ab06-4b2d-a9c4-33c90b7f2070 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:42.912739 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27227285-ab06-4b2d-a9c4-33c90b7f2070 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:42.916363 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 906/1790] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:42 2022] POST /identity/v3/users => generated 301 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:42.916481 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27227285-ab06-4b2d-a9c4-33c90b7f2070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:42.916558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf76a5-0590-4d15-a9d1-966e1c43a4e4 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:42.916639 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27227285-ab06-4b2d-a9c4-33c90b7f2070 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:42.925156 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 885/1791] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:42 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:42.925367 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf76a5-0590-4d15-a9d1-966e1c43a4e4 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:42.925839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27227285-ab06-4b2d-a9c4-33c90b7f2070 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:42.930490 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d77c5-7a6a-4263-88ce-a57b0a65cd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:42.931004 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d77c5-7a6a-4263-88ce-a57b0a65cd23 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:42.931326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d77c5-7a6a-4263-88ce-a57b0a65cd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:42.931595 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d77c5-7a6a-4263-88ce-a57b0a65cd23 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:42.932311 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d77c5-7a6a-4263-88ce-a57b0a65cd23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:42.933919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d77c5-7a6a-4263-88ce-a57b0a65cd23 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:42.934963 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27227285-ab06-4b2d-a9c4-33c90b7f2070 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:42.939665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9916912-b339-4331-b857-2fb21da9a6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:42.940302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9916912-b339-4331-b857-2fb21da9a6b3 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:42.940585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9916912-b339-4331-b857-2fb21da9a6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:42.941022 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9916912-b339-4331-b857-2fb21da9a6b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:42.941737 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9916912-b339-4331-b857-2fb21da9a6b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:42.943324 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9916912-b339-4331-b857-2fb21da9a6b3 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:42.947046 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 907/1792] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:42.947046 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d77c5-7a6a-4263-88ce-a57b0a65cd23 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:42.955933 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 886/1793] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:42 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:42.955933 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9916912-b339-4331-b857-2fb21da9a6b3 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:42.956277 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d77c5-7a6a-4263-88ce-a57b0a65cd23 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:42.961215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af78581-fc8a-4b74-b51b-74af84a24707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:42.961781 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af78581-fc8a-4b74-b51b-74af84a24707 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:42.962063 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af78581-fc8a-4b74-b51b-74af84a24707 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:42.962332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af78581-fc8a-4b74-b51b-74af84a24707 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:42.963050 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af78581-fc8a-4b74-b51b-74af84a24707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:42.963471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9916912-b339-4331-b857-2fb21da9a6b3 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:42.964626 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af78581-fc8a-4b74-b51b-74af84a24707 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:42.968056 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb4a5b-2963-4190-bda3-402716f5f429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:42.968945 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb4a5b-2963-4190-bda3-402716f5f429 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:42.969220 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb4a5b-2963-4190-bda3-402716f5f429 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:42.969487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb4a5b-2963-4190-bda3-402716f5f429 admin admin] PATH_INFO: `/v3/projects/3357719665a946418154cd6596f4c71d/users/38a2a8c04b26416f97b2addba283fc68/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:42.971446 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb4a5b-2963-4190-bda3-402716f5f429 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3357719665a946418154cd6596f4c71d, user_id=38a2a8c04b26416f97b2addba283fc68, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:42.974089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb4a5b-2963-4190-bda3-402716f5f429 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:42.975257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 908/1794] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:42.976226 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af78581-fc8a-4b74-b51b-74af84a24707 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:42.985310 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af78581-fc8a-4b74-b51b-74af84a24707 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:42.986439 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 887/1795] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:42 2022] PUT /identity/v3/projects/3357719665a946418154cd6596f4c71d/users/38a2a8c04b26416f97b2addba283fc68/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:42.987248 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb4a5b-2963-4190-bda3-402716f5f429 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:43.043775 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b652483-7631-4314-886d-3759d99c0f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:43.044505 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b652483-7631-4314-886d-3759d99c0f67 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:43.044751 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b652483-7631-4314-886d-3759d99c0f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:43.045024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b652483-7631-4314-886d-3759d99c0f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:43.045778 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b652483-7631-4314-886d-3759d99c0f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:43.047432 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b652483-7631-4314-886d-3759d99c0f67 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:43.047873 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb4a5b-2963-4190-bda3-402716f5f429 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:43.052685 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02857b39-e8ad-4245-a277-db650895eb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:43.053568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02857b39-e8ad-4245-a277-db650895eb02 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:43.053843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02857b39-e8ad-4245-a277-db650895eb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:43.054110 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02857b39-e8ad-4245-a277-db650895eb02 admin admin] PATH_INFO: `/v3/projects/b53f4e48011e4f019fa99c1ac0dd307e/users/d1030b74c60c4a38bb563f69c3474e0e/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:43.057578 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 909/1796] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:43.058726 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08621d1b-bcbf-4aab-884b-970e05490338 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:43.059005 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08621d1b-bcbf-4aab-884b-970e05490338 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:43.059295 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08621d1b-bcbf-4aab-884b-970e05490338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:43.071510 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02857b39-e8ad-4245-a277-db650895eb02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b53f4e48011e4f019fa99c1ac0dd307e, user_id=d1030b74c60c4a38bb563f69c3474e0e, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:43.074227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02857b39-e8ad-4245-a277-db650895eb02 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:43.087330 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-08621d1b-bcbf-4aab-884b-970e05490338 None None] MFA Rules not processed for user `38a2a8c04b26416f97b2addba283fc68`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:43.089142 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-08621d1b-bcbf-4aab-884b-970e05490338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:43.092314 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 888/1797] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:42 2022] PUT /identity/v3/projects/b53f4e48011e4f019fa99c1ac0dd307e/users/d1030b74c60c4a38bb563f69c3474e0e/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:43.093248 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02857b39-e8ad-4245-a277-db650895eb02 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:43.198809 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02857b39-e8ad-4245-a277-db650895eb02 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:43.204425 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121360b7-6ca4-4b82-a187-fbda268f3600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:43.205107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 910/1798] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:43 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:43.206953 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42c51a-835f-41f7-9618-d7b39f18b927 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:43.207411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42c51a-835f-41f7-9618-d7b39f18b927 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:43.207726 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42c51a-835f-41f7-9618-d7b39f18b927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:43.208853 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121360b7-6ca4-4b82-a187-fbda268f3600 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:43.209226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121360b7-6ca4-4b82-a187-fbda268f3600 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:43.209516 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121360b7-6ca4-4b82-a187-fbda268f3600 admin admin] PATH_INFO: `/v3/projects/1f6b9be562344f37b2318ef3201e10c1/users/87bad88fd9594220881c269df4490e61/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:43.227631 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121360b7-6ca4-4b82-a187-fbda268f3600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f6b9be562344f37b2318ef3201e10c1, user_id=87bad88fd9594220881c269df4490e61, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:43.230455 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121360b7-6ca4-4b82-a187-fbda268f3600 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:43.242116 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-5d42c51a-835f-41f7-9618-d7b39f18b927 None None] MFA Rules not processed for user `d1030b74c60c4a38bb563f69c3474e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:43.243468 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-5d42c51a-835f-41f7-9618-d7b39f18b927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:43.244388 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 889/1799] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:43 2022] PUT /identity/v3/projects/1f6b9be562344f37b2318ef3201e10c1/users/87bad88fd9594220881c269df4490e61/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:43.253483 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121360b7-6ca4-4b82-a187-fbda268f3600 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:43.253483 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-121360b7-6ca4-4b82-a187-fbda268f3600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:43.307186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121360b7-6ca4-4b82-a187-fbda268f3600 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:43.310821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e95a7-1d92-4958-b241-e8a27a4ddf49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:43.311756 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e95a7-1d92-4958-b241-e8a27a4ddf49 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:43.312014 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e95a7-1d92-4958-b241-e8a27a4ddf49 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:43.312417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e95a7-1d92-4958-b241-e8a27a4ddf49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:43.318988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 911/1800] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:43 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:43.320949 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdfc4f-8c83-4ede-8dc3-f8eb5df752fc None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:43.321432 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdfc4f-8c83-4ede-8dc3-f8eb5df752fc None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:43.321891 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdfc4f-8c83-4ede-8dc3-f8eb5df752fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:43.351654 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-1bbdfc4f-8c83-4ede-8dc3-f8eb5df752fc None None] MFA Rules not processed for user `87bad88fd9594220881c269df4490e61`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:43.352891 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-1bbdfc4f-8c83-4ede-8dc3-f8eb5df752fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:43.382081 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e95a7-1d92-4958-b241-e8a27a4ddf49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:43.383698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e95a7-1d92-4958-b241-e8a27a4ddf49 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:43.393432 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 890/1801] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:43 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:43.394188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e95a7-1d92-4958-b241-e8a27a4ddf49 service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:43.430472 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 912/1802] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:43.434603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdfc4f-8c83-4ede-8dc3-f8eb5df752fc None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:43.470641 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e95a7-1d92-4958-b241-e8a27a4ddf49 service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:43.496464 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d72483-3f44-4224-8b75-0f0d2fe69534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:43.497654 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d72483-3f44-4224-8b75-0f0d2fe69534 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:43.497654 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d72483-3f44-4224-8b75-0f0d2fe69534 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:43.497885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d72483-3f44-4224-8b75-0f0d2fe69534 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:43.498629 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d72483-3f44-4224-8b75-0f0d2fe69534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:43.501038 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d72483-3f44-4224-8b75-0f0d2fe69534 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:43.555257 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 891/1803] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:43 2022] POST /identity/v3/projects => generated 357 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:43.557012 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d72483-3f44-4224-8b75-0f0d2fe69534 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:43.591318 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdfc4f-8c83-4ede-8dc3-f8eb5df752fc None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:43.657443 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd375d9-8e2c-497b-843d-5caf3acada64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:43.658126 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd375d9-8e2c-497b-843d-5caf3acada64 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:43.658463 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd375d9-8e2c-497b-843d-5caf3acada64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:43.658741 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd375d9-8e2c-497b-843d-5caf3acada64 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:43.659447 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd375d9-8e2c-497b-843d-5caf3acada64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:43.663769 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd375d9-8e2c-497b-843d-5caf3acada64 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:43.683489 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d72483-3f44-4224-8b75-0f0d2fe69534 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:43.688913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4336be-bde2-4b03-a2a2-c5bb5bcc15e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:43.689658 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4336be-bde2-4b03-a2a2-c5bb5bcc15e0 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:43.690144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4336be-bde2-4b03-a2a2-c5bb5bcc15e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:43.690499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4336be-bde2-4b03-a2a2-c5bb5bcc15e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:43.732435 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 913/1804] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:43 2022] POST /identity/v3/projects => generated 399 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:47:43.733519 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd375d9-8e2c-497b-843d-5caf3acada64 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:43.810971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4336be-bde2-4b03-a2a2-c5bb5bcc15e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:43.812875 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4336be-bde2-4b03-a2a2-c5bb5bcc15e0 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:43.819644 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd375d9-8e2c-497b-843d-5caf3acada64 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:43.823670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 892/1805] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:43.824575 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4336be-bde2-4b03-a2a2-c5bb5bcc15e0 service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:43.825994 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d3473-3dac-4bf6-9d46-d3b59471aa10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:43.827489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d3473-3dac-4bf6-9d46-d3b59471aa10 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:43.828502 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d3473-3dac-4bf6-9d46-d3b59471aa10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:43.829177 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d3473-3dac-4bf6-9d46-d3b59471aa10 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:43.830071 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d3473-3dac-4bf6-9d46-d3b59471aa10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:43.832333 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d3473-3dac-4bf6-9d46-d3b59471aa10 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:43.836639 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4336be-bde2-4b03-a2a2-c5bb5bcc15e0 service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:43.847857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86319413-0070-42c2-9038-e4850fecc728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:43.848720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86319413-0070-42c2-9038-e4850fecc728 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:43.849078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86319413-0070-42c2-9038-e4850fecc728 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:43.849419 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86319413-0070-42c2-9038-e4850fecc728 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:43.850355 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86319413-0070-42c2-9038-e4850fecc728 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:43.852602 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86319413-0070-42c2-9038-e4850fecc728 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:43.873986 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 914/1806] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:43 2022] POST /identity/v3/users => generated 285 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:43.876496 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d3473-3dac-4bf6-9d46-d3b59471aa10 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:43.896742 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 893/1807] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:43 2022] POST /identity/v3/users => generated 306 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:43.897915 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86319413-0070-42c2-9038-e4850fecc728 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:43.900952 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d3473-3dac-4bf6-9d46-d3b59471aa10 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:43.914753 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86319413-0070-42c2-9038-e4850fecc728 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:43.917077 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ab5be-e67f-4daa-be18-51321bb708fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:43.917783 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ab5be-e67f-4daa-be18-51321bb708fe admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:43.918045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ab5be-e67f-4daa-be18-51321bb708fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:43.918335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ab5be-e67f-4daa-be18-51321bb708fe admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:43.919054 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6ab5be-e67f-4daa-be18-51321bb708fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:43.922331 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6ab5be-e67f-4daa-be18-51321bb708fe admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:43.926686 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd05cb9-c005-425c-99b2-b3c669c99af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:43.928000 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd05cb9-c005-425c-99b2-b3c669c99af1 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:43.928423 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd05cb9-c005-425c-99b2-b3c669c99af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:43.928735 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd05cb9-c005-425c-99b2-b3c669c99af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:43.929505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd05cb9-c005-425c-99b2-b3c669c99af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:43.931869 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd05cb9-c005-425c-99b2-b3c669c99af1 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:43.944560 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 894/1808] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:43.951584 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 915/1809] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:47:43.955123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd05cb9-c005-425c-99b2-b3c669c99af1 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:43.956895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ab5be-e67f-4daa-be18-51321bb708fe admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:43.981317 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ab5be-e67f-4daa-be18-51321bb708fe admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:43.983389 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd05cb9-c005-425c-99b2-b3c669c99af1 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:43.992468 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecb239-c0e0-4d2d-b6e0-7ef62eaa5ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:43.993883 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecb239-c0e0-4d2d-b6e0-7ef62eaa5ef6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:43.994294 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecb239-c0e0-4d2d-b6e0-7ef62eaa5ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:43.994911 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecb239-c0e0-4d2d-b6e0-7ef62eaa5ef6 admin admin] PATH_INFO: `/v3/projects/274372d2bda549dea5145b84ead70ee8/users/805b6824405444dc807de044dafe4452/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:44.000333 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b153f58-0369-40ed-8613-4fb65db861fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:44.001606 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b153f58-0369-40ed-8613-4fb65db861fe admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:44.009103 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b153f58-0369-40ed-8613-4fb65db861fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:44.009625 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b153f58-0369-40ed-8613-4fb65db861fe admin admin] PATH_INFO: `/v3/projects/5a1114401b21495da8f9d02cd31146c8/users/589a9be4be034a00b1b8d75923b8088e/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:44.023841 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ecb239-c0e0-4d2d-b6e0-7ef62eaa5ef6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=274372d2bda549dea5145b84ead70ee8, user_id=805b6824405444dc807de044dafe4452, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:44.027881 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b153f58-0369-40ed-8613-4fb65db861fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a1114401b21495da8f9d02cd31146c8, user_id=589a9be4be034a00b1b8d75923b8088e, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:44.030909 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b153f58-0369-40ed-8613-4fb65db861fe admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:44.033774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ecb239-c0e0-4d2d-b6e0-7ef62eaa5ef6 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:44.053517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 916/1810] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:43 2022] PUT /identity/v3/projects/5a1114401b21495da8f9d02cd31146c8/users/589a9be4be034a00b1b8d75923b8088e/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:44.059043 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b153f58-0369-40ed-8613-4fb65db861fe admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:44.067733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 895/1811] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:43 2022] PUT /identity/v3/projects/274372d2bda549dea5145b84ead70ee8/users/805b6824405444dc807de044dafe4452/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:44.071871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecb239-c0e0-4d2d-b6e0-7ef62eaa5ef6 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:44.175953 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b153f58-0369-40ed-8613-4fb65db861fe admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:44.180813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599f0b3-89cb-4b7d-8845-af40ffa8b411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:44.181480 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599f0b3-89cb-4b7d-8845-af40ffa8b411 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:44.181757 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599f0b3-89cb-4b7d-8845-af40ffa8b411 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:44.182370 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599f0b3-89cb-4b7d-8845-af40ffa8b411 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:44.182844 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c599f0b3-89cb-4b7d-8845-af40ffa8b411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:44.184010 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecb239-c0e0-4d2d-b6e0-7ef62eaa5ef6 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:44.184442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c599f0b3-89cb-4b7d-8845-af40ffa8b411 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:44.188784 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350b2ff-8bce-41c6-bcd5-5599f47304f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:44.189487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350b2ff-8bce-41c6-bcd5-5599f47304f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:44.189785 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350b2ff-8bce-41c6-bcd5-5599f47304f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:44.190070 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350b2ff-8bce-41c6-bcd5-5599f47304f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:44.190997 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f350b2ff-8bce-41c6-bcd5-5599f47304f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:44.192868 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f350b2ff-8bce-41c6-bcd5-5599f47304f3 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:44.196746 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 917/1812] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:44.200316 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599f0b3-89cb-4b7d-8845-af40ffa8b411 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:44.208071 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 896/1813] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:47:44.211255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599f0b3-89cb-4b7d-8845-af40ffa8b411 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:44.213280 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350b2ff-8bce-41c6-bcd5-5599f47304f3 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:44.228582 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350b2ff-8bce-41c6-bcd5-5599f47304f3 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:44.232276 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098bc5b-c1c4-4653-b238-a8c0d5cd6923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:44.236709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2dcf23-7924-441d-9a32-280ecad36713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:44.237308 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098bc5b-c1c4-4653-b238-a8c0d5cd6923 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:44.237652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098bc5b-c1c4-4653-b238-a8c0d5cd6923 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:44.237934 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098bc5b-c1c4-4653-b238-a8c0d5cd6923 admin admin] PATH_INFO: `/v3/projects/5a1114401b21495da8f9d02cd31146c8/users/589a9be4be034a00b1b8d75923b8088e/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:44.238429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2dcf23-7924-441d-9a32-280ecad36713 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:44.241010 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2dcf23-7924-441d-9a32-280ecad36713 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:44.241507 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2dcf23-7924-441d-9a32-280ecad36713 admin admin] PATH_INFO: `/v3/projects/274372d2bda549dea5145b84ead70ee8/users/805b6824405444dc807de044dafe4452/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:44.243623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2dcf23-7924-441d-9a32-280ecad36713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=274372d2bda549dea5145b84ead70ee8, user_id=805b6824405444dc807de044dafe4452, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:44.246460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098bc5b-c1c4-4653-b238-a8c0d5cd6923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a1114401b21495da8f9d02cd31146c8, user_id=589a9be4be034a00b1b8d75923b8088e, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:44.246593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2dcf23-7924-441d-9a32-280ecad36713 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:44.249979 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098bc5b-c1c4-4653-b238-a8c0d5cd6923 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:44.262015 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 897/1814] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:44 2022] PUT /identity/v3/projects/274372d2bda549dea5145b84ead70ee8/users/805b6824405444dc807de044dafe4452/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:47:44.273785 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee31db7-cc60-45fd-a33e-4bff1115fe7e None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:44.273785 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee31db7-cc60-45fd-a33e-4bff1115fe7e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:44.273785 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee31db7-cc60-45fd-a33e-4bff1115fe7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:44.274491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 918/1815] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:44 2022] PUT /identity/v3/projects/5a1114401b21495da8f9d02cd31146c8/users/589a9be4be034a00b1b8d75923b8088e/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:47:44.286760 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c03c5-591a-4d7b-9cac-b55126fac219 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:44.287039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c03c5-591a-4d7b-9cac-b55126fac219 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:44.287318 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c03c5-591a-4d7b-9cac-b55126fac219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:44.314955 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-3ee31db7-cc60-45fd-a33e-4bff1115fe7e None None] MFA Rules not processed for user `805b6824405444dc807de044dafe4452`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:44.316107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-1a8c03c5-591a-4d7b-9cac-b55126fac219 None None] MFA Rules not processed for user `589a9be4be034a00b1b8d75923b8088e`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:44.316394 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-3ee31db7-cc60-45fd-a33e-4bff1115fe7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:44.317477 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-1a8c03c5-591a-4d7b-9cac-b55126fac219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:44.389956 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 919/1816] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:44 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:47:44.390832 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 898/1817] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:44 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:44.401837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c03c5-591a-4d7b-9cac-b55126fac219 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:44.403335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-1a8c03c5-591a-4d7b-9cac-b55126fac219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:44.405882 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee31db7-cc60-45fd-a33e-4bff1115fe7e None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:44.459931 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c03c5-591a-4d7b-9cac-b55126fac219 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:44.460200 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee31db7-cc60-45fd-a33e-4bff1115fe7e None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:44.463723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb9fd0-55e0-4460-98df-b28993211ac2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:44.463957 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5d7c3-6683-4217-9c47-7ddf03510bfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:44.464369 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb9fd0-55e0-4460-98df-b28993211ac2 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:44.464601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5d7c3-6683-4217-9c47-7ddf03510bfe service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:44.464734 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb9fd0-55e0-4460-98df-b28993211ac2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:44.464919 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5d7c3-6683-4217-9c47-7ddf03510bfe service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:44.465017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb9fd0-55e0-4460-98df-b28993211ac2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:44.465187 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5d7c3-6683-4217-9c47-7ddf03510bfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:44.474694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d5d7c3-6683-4217-9c47-7ddf03510bfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:44.475486 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb9fd0-55e0-4460-98df-b28993211ac2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:44.476149 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d5d7c3-6683-4217-9c47-7ddf03510bfe service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:44.476262 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb9fd0-55e0-4460-98df-b28993211ac2 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:44.486656 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 899/1818] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:44 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:44.488589 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 920/1819] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:44 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:44.585265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5d7c3-6683-4217-9c47-7ddf03510bfe service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:44.593912 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5d7c3-6683-4217-9c47-7ddf03510bfe service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:44.597503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce0bc1-7832-4596-8655-b6ad7b2a2fc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:44.598136 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce0bc1-7832-4596-8655-b6ad7b2a2fc2 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:44.598394 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb9fd0-55e0-4460-98df-b28993211ac2 service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:44.598506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce0bc1-7832-4596-8655-b6ad7b2a2fc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:44.598713 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce0bc1-7832-4596-8655-b6ad7b2a2fc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:44.607230 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb9fd0-55e0-4460-98df-b28993211ac2 service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:44.611599 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fc26a-7ec4-4335-ab21-bcaab26f3660 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:44.612277 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fc26a-7ec4-4335-ab21-bcaab26f3660 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:44.612554 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fc26a-7ec4-4335-ab21-bcaab26f3660 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:44.612908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fc26a-7ec4-4335-ab21-bcaab26f3660 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:44.659142 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce0bc1-7832-4596-8655-b6ad7b2a2fc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:44.661560 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce0bc1-7832-4596-8655-b6ad7b2a2fc2 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:44.671459 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 900/1820] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:44 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 09:47:44.674333 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34fc26a-7ec4-4335-ab21-bcaab26f3660 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:44.675796 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34fc26a-7ec4-4335-ab21-bcaab26f3660 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:44.685416 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 921/1821] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:44 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 09:47:48.772884 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce0bc1-7832-4596-8655-b6ad7b2a2fc2 service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:48.887065 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce0bc1-7832-4596-8655-b6ad7b2a2fc2 service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:48.892579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a3b91-a7c3-408c-961d-3b6b2ebb3dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:48.893533 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a3b91-a7c3-408c-961d-3b6b2ebb3dd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:48.893852 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a3b91-a7c3-408c-961d-3b6b2ebb3dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:48.894290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a3b91-a7c3-408c-961d-3b6b2ebb3dd8 admin admin] PATH_INFO: `/v3/users/c54d3b25afc74c13a9f6b85ab567fe0a` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:48.895615 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a3b91-a7c3-408c-961d-3b6b2ebb3dd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c54d3b25afc74c13a9f6b85ab567fe0a)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:48.898393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a3b91-a7c3-408c-961d-3b6b2ebb3dd8 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:48.948086 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-dc0a3b91-a7c3-408c-961d-3b6b2ebb3dd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c54d3b25afc74c13a9f6b85ab567fe0a'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:48.955027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-dc0a3b91-a7c3-408c-961d-3b6b2ebb3dd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c54d3b25afc74c13a9f6b85ab567fe0a'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:48.965536 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-dc0a3b91-a7c3-408c-961d-3b6b2ebb3dd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c54d3b25afc74c13a9f6b85ab567fe0a'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:48.966356 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-dc0a3b91-a7c3-408c-961d-3b6b2ebb3dd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c54d3b25afc74c13a9f6b85ab567fe0a'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:48.967171 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-dc0a3b91-a7c3-408c-961d-3b6b2ebb3dd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c54d3b25afc74c13a9f6b85ab567fe0a'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:48.986155 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 901/1822] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:47:48 2022] DELETE /identity/v3/users/c54d3b25afc74c13a9f6b85ab567fe0a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:48.990627 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b70b8-a3a9-4d18-acbf-098ad65eec2f None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:48.991214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b70b8-a3a9-4d18-acbf-098ad65eec2f None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:48.991214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b70b8-a3a9-4d18-acbf-098ad65eec2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:49.034630 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-247b70b8-a3a9-4d18-acbf-098ad65eec2f None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:49.035879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-247b70b8-a3a9-4d18-acbf-098ad65eec2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:49.139522 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 922/1823] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:49.164334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a3b91-a7c3-408c-961d-3b6b2ebb3dd8 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:49.166548 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-dc0a3b91-a7c3-408c-961d-3b6b2ebb3dd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:49.275658 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a3b91-a7c3-408c-961d-3b6b2ebb3dd8 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:49.286037 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab4479-1886-45aa-8d23-6efdd57d74d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:49.286696 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab4479-1886-45aa-8d23-6efdd57d74d0 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:49.286982 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab4479-1886-45aa-8d23-6efdd57d74d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:49.287720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab4479-1886-45aa-8d23-6efdd57d74d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:49.298193 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ab4479-1886-45aa-8d23-6efdd57d74d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:49.299670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ab4479-1886-45aa-8d23-6efdd57d74d0 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:49.313108 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 902/1824] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:49.779279 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b70b8-a3a9-4d18-acbf-098ad65eec2f None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:49.780830 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-247b70b8-a3a9-4d18-acbf-098ad65eec2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:49.797311 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b70b8-a3a9-4d18-acbf-098ad65eec2f None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:49.803444 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc92e5b-2402-4784-810a-1779f789dcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:49.804695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc92e5b-2402-4784-810a-1779f789dcb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:49.805238 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc92e5b-2402-4784-810a-1779f789dcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:49.805711 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc92e5b-2402-4784-810a-1779f789dcb4 admin admin] PATH_INFO: `/v3/projects/fc1d9b49a83f4b30abb84e6727f44d76` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:49.807359 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc92e5b-2402-4784-810a-1779f789dcb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc1d9b49a83f4b30abb84e6727f44d76)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:49.809869 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc92e5b-2402-4784-810a-1779f789dcb4 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:49.888237 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8dc92e5b-2402-4784-810a-1779f789dcb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc1d9b49a83f4b30abb84e6727f44d76', 'request_id': 'req-8dc92e5b-2402-4784-810a-1779f789dcb4'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:49.897168 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8dc92e5b-2402-4784-810a-1779f789dcb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc1d9b49a83f4b30abb84e6727f44d76', 'request_id': 'req-8dc92e5b-2402-4784-810a-1779f789dcb4'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:49.902624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8dc92e5b-2402-4784-810a-1779f789dcb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc1d9b49a83f4b30abb84e6727f44d76', 'request_id': 'req-8dc92e5b-2402-4784-810a-1779f789dcb4'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:49.939585 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8dc92e5b-2402-4784-810a-1779f789dcb4 admin admin] The token cache is being invalidate because project fc1d9b49a83f4b30abb84e6727f44d76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:47:49.939585 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8dc92e5b-2402-4784-810a-1779f789dcb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:49.939834 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8dc92e5b-2402-4784-810a-1779f789dcb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:49.941687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 923/1825] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:47:49 2022] DELETE /identity/v3/projects/fc1d9b49a83f4b30abb84e6727f44d76 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:49.946121 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab4479-1886-45aa-8d23-6efdd57d74d0 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:49.961437 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab4479-1886-45aa-8d23-6efdd57d74d0 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:50.023218 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07874e-3c52-4dcf-b5b6-9efb39a694a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:50.024284 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07874e-3c52-4dcf-b5b6-9efb39a694a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:50.024565 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07874e-3c52-4dcf-b5b6-9efb39a694a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:50.024872 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07874e-3c52-4dcf-b5b6-9efb39a694a1 admin admin] PATH_INFO: `/v3/users/38a2a8c04b26416f97b2addba283fc68` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:50.026366 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b07874e-3c52-4dcf-b5b6-9efb39a694a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38a2a8c04b26416f97b2addba283fc68)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:50.028546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b07874e-3c52-4dcf-b5b6-9efb39a694a1 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:50.076709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0b07874e-3c52-4dcf-b5b6-9efb39a694a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38a2a8c04b26416f97b2addba283fc68'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:50.084410 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0b07874e-3c52-4dcf-b5b6-9efb39a694a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38a2a8c04b26416f97b2addba283fc68'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:50.094583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0b07874e-3c52-4dcf-b5b6-9efb39a694a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38a2a8c04b26416f97b2addba283fc68'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:50.095213 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0b07874e-3c52-4dcf-b5b6-9efb39a694a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38a2a8c04b26416f97b2addba283fc68'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:50.095859 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0b07874e-3c52-4dcf-b5b6-9efb39a694a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38a2a8c04b26416f97b2addba283fc68'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:50.115219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 903/1826] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:47:49 2022] DELETE /identity/v3/users/38a2a8c04b26416f97b2addba283fc68 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:47:50.410139 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc92e5b-2402-4784-810a-1779f789dcb4 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:50.411612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-8dc92e5b-2402-4784-810a-1779f789dcb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:50.426908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc92e5b-2402-4784-810a-1779f789dcb4 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:50.484608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72880b0f-0451-4927-b794-c1ae6870e206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:50.485497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72880b0f-0451-4927-b794-c1ae6870e206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:50.486073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72880b0f-0451-4927-b794-c1ae6870e206 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:50.486350 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72880b0f-0451-4927-b794-c1ae6870e206 admin admin] PATH_INFO: `/v3/projects/3357719665a946418154cd6596f4c71d` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:50.487584 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72880b0f-0451-4927-b794-c1ae6870e206 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3357719665a946418154cd6596f4c71d)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:50.489523 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72880b0f-0451-4927-b794-c1ae6870e206 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:50.544033 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-72880b0f-0451-4927-b794-c1ae6870e206 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3357719665a946418154cd6596f4c71d', 'request_id': 'req-72880b0f-0451-4927-b794-c1ae6870e206'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:50.550826 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-72880b0f-0451-4927-b794-c1ae6870e206 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3357719665a946418154cd6596f4c71d', 'request_id': 'req-72880b0f-0451-4927-b794-c1ae6870e206'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:50.557310 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-72880b0f-0451-4927-b794-c1ae6870e206 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3357719665a946418154cd6596f4c71d', 'request_id': 'req-72880b0f-0451-4927-b794-c1ae6870e206'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:50.573551 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-72880b0f-0451-4927-b794-c1ae6870e206 admin admin] The token cache is being invalidate because project 3357719665a946418154cd6596f4c71d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:47:50.573893 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-72880b0f-0451-4927-b794-c1ae6870e206 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:50.574695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-72880b0f-0451-4927-b794-c1ae6870e206 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:50.576358 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 924/1827] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:47:50 2022] DELETE /identity/v3/projects/3357719665a946418154cd6596f4c71d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:50.582563 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046add52-bb8e-481c-b895-7e429a01f6a9 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:50.582848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046add52-bb8e-481c-b895-7e429a01f6a9 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:50.583123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046add52-bb8e-481c-b895-7e429a01f6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:50.600005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-046add52-bb8e-481c-b895-7e429a01f6a9 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:50.601209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-046add52-bb8e-481c-b895-7e429a01f6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:50.659811 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 904/1828] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:47:50.667895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134e94d-fccf-4a04-b9fb-671769bbf81b None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:50.668176 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134e94d-fccf-4a04-b9fb-671769bbf81b None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:50.668450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134e94d-fccf-4a04-b9fb-671769bbf81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:50.686276 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-2134e94d-fccf-4a04-b9fb-671769bbf81b None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:50.687456 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-2134e94d-fccf-4a04-b9fb-671769bbf81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:50.695380 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 925/1829] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:50.699657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046add52-bb8e-481c-b895-7e429a01f6a9 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:50.709675 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046add52-bb8e-481c-b895-7e429a01f6a9 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:50.715092 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfb82f-ce61-48d9-aa9b-8e6ba9dcfc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:50.715771 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfb82f-ce61-48d9-aa9b-8e6ba9dcfc76 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:50.716043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfb82f-ce61-48d9-aa9b-8e6ba9dcfc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:50.716311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfb82f-ce61-48d9-aa9b-8e6ba9dcfc76 admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:50.717071 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfb82f-ce61-48d9-aa9b-8e6ba9dcfc76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:47:50.717366 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfb82f-ce61-48d9-aa9b-8e6ba9dcfc76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:50.719032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfb82f-ce61-48d9-aa9b-8e6ba9dcfc76 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:50.731320 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 905/1830] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:47:50.736570 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134e94d-fccf-4a04-b9fb-671769bbf81b None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:50.747641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134e94d-fccf-4a04-b9fb-671769bbf81b None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:50.752640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c85c02a-6ffb-4b15-b42b-90fdcc835151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:50.753346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c85c02a-6ffb-4b15-b42b-90fdcc835151 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:50.753584 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c85c02a-6ffb-4b15-b42b-90fdcc835151 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:50.753862 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c85c02a-6ffb-4b15-b42b-90fdcc835151 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:50.754669 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c85c02a-6ffb-4b15-b42b-90fdcc835151 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:50.756548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c85c02a-6ffb-4b15-b42b-90fdcc835151 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:50.783482 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 926/1831] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:50 2022] POST /identity/v3/projects => generated 359 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:47:50.787493 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfb82f-ce61-48d9-aa9b-8e6ba9dcfc76 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:50.797445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfb82f-ce61-48d9-aa9b-8e6ba9dcfc76 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:50.853235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181952b8-a03e-4564-b843-8be7ff440f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:50.853900 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181952b8-a03e-4564-b843-8be7ff440f90 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:50.854173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181952b8-a03e-4564-b843-8be7ff440f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:50.854440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181952b8-a03e-4564-b843-8be7ff440f90 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:50.855141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181952b8-a03e-4564-b843-8be7ff440f90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:50.857097 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181952b8-a03e-4564-b843-8be7ff440f90 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:50.877812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 906/1832] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:50 2022] POST /identity/v3/users => generated 280 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:50.882194 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c85c02a-6ffb-4b15-b42b-90fdcc835151 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:50.893004 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c85c02a-6ffb-4b15-b42b-90fdcc835151 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:50.898401 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b9f4a-d97b-487e-ac04-47c589bc214c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:50.899070 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b9f4a-d97b-487e-ac04-47c589bc214c admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:50.899342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b9f4a-d97b-487e-ac04-47c589bc214c admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:50.899614 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b9f4a-d97b-487e-ac04-47c589bc214c admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:50.900384 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b9f4a-d97b-487e-ac04-47c589bc214c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:50.902557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b9f4a-d97b-487e-ac04-47c589bc214c admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:50.914150 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 927/1833] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:50.918361 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181952b8-a03e-4564-b843-8be7ff440f90 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:50.930741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181952b8-a03e-4564-b843-8be7ff440f90 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:50.935721 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d193b-88c3-4d3b-8ef2-3e1c0ef92caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:50.936446 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d193b-88c3-4d3b-8ef2-3e1c0ef92caf admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:50.936750 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d193b-88c3-4d3b-8ef2-3e1c0ef92caf admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:50.937055 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d193b-88c3-4d3b-8ef2-3e1c0ef92caf admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:50.937816 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2d193b-88c3-4d3b-8ef2-3e1c0ef92caf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:50.939393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2d193b-88c3-4d3b-8ef2-3e1c0ef92caf admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:50.952836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 907/1834] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:50.954050 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b9f4a-d97b-487e-ac04-47c589bc214c admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:50.964359 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b9f4a-d97b-487e-ac04-47c589bc214c admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:50.969055 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f1732-3840-469e-9839-aa1f56f60243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:50.969934 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f1732-3840-469e-9839-aa1f56f60243 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:50.970208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f1732-3840-469e-9839-aa1f56f60243 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:50.970473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f1732-3840-469e-9839-aa1f56f60243 admin admin] PATH_INFO: `/v3/projects/24e1f6a786ee43dead5baf7395538b88/users/5aa9f322701c44c08edd033a1ed933d2/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:50.984471 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9f1732-3840-469e-9839-aa1f56f60243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24e1f6a786ee43dead5baf7395538b88, user_id=5aa9f322701c44c08edd033a1ed933d2, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:50.987459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9f1732-3840-469e-9839-aa1f56f60243 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:51.000985 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 928/1835] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:50 2022] PUT /identity/v3/projects/24e1f6a786ee43dead5baf7395538b88/users/5aa9f322701c44c08edd033a1ed933d2/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:47:51.012182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681db1a-e9a4-4c05-bbbf-07d33431a425 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:51.012460 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681db1a-e9a4-4c05-bbbf-07d33431a425 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:51.012760 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681db1a-e9a4-4c05-bbbf-07d33431a425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:51.038842 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-b681db1a-e9a4-4c05-bbbf-07d33431a425 None None] MFA Rules not processed for user `5aa9f322701c44c08edd033a1ed933d2`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:51.040082 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-b681db1a-e9a4-4c05-bbbf-07d33431a425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:51.104382 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 908/1836] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:51 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:47:51.137801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f1732-3840-469e-9839-aa1f56f60243 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:51.139856 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-ad9f1732-3840-469e-9839-aa1f56f60243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:51.205726 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f1732-3840-469e-9839-aa1f56f60243 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:51.209737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb5ba4-aa4f-4ae5-af0b-00412f60a042 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:51.210474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb5ba4-aa4f-4ae5-af0b-00412f60a042 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:51.210746 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb5ba4-aa4f-4ae5-af0b-00412f60a042 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:51.211269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb5ba4-aa4f-4ae5-af0b-00412f60a042 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:51.219551 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcb5ba4-aa4f-4ae5-af0b-00412f60a042 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:51.221038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcb5ba4-aa4f-4ae5-af0b-00412f60a042 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:51.229630 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 929/1837] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:51 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 09:47:51.244329 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681db1a-e9a4-4c05-bbbf-07d33431a425 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:51.306602 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681db1a-e9a4-4c05-bbbf-07d33431a425 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:51.312237 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50262cd0-2091-4d2d-8757-1398fd50981b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:51.313132 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50262cd0-2091-4d2d-8757-1398fd50981b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:51.313408 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50262cd0-2091-4d2d-8757-1398fd50981b admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:51.313676 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50262cd0-2091-4d2d-8757-1398fd50981b admin admin] PATH_INFO: `/v3/users/5aa9f322701c44c08edd033a1ed933d2` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:51.314944 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50262cd0-2091-4d2d-8757-1398fd50981b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aa9f322701c44c08edd033a1ed933d2)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:51.316889 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50262cd0-2091-4d2d-8757-1398fd50981b admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:51.369757 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-50262cd0-2091-4d2d-8757-1398fd50981b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aa9f322701c44c08edd033a1ed933d2'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:51.377328 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-50262cd0-2091-4d2d-8757-1398fd50981b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aa9f322701c44c08edd033a1ed933d2'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:51.386436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-50262cd0-2091-4d2d-8757-1398fd50981b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aa9f322701c44c08edd033a1ed933d2'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:51.387041 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-50262cd0-2091-4d2d-8757-1398fd50981b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aa9f322701c44c08edd033a1ed933d2'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:51.387649 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-50262cd0-2091-4d2d-8757-1398fd50981b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aa9f322701c44c08edd033a1ed933d2'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:51.405172 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 909/1838] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:47:51 2022] DELETE /identity/v3/users/5aa9f322701c44c08edd033a1ed933d2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:51.409770 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897d858-f95f-48b8-bbe5-d1bef19af404 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:51.410133 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897d858-f95f-48b8-bbe5-d1bef19af404 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:51.410472 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897d858-f95f-48b8-bbe5-d1bef19af404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:51.428327 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-2897d858-f95f-48b8-bbe5-d1bef19af404 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:51.429575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-2897d858-f95f-48b8-bbe5-d1bef19af404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:51.487334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 930/1839] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:51.497210 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50262cd0-2091-4d2d-8757-1398fd50981b admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:51.499119 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-50262cd0-2091-4d2d-8757-1398fd50981b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:51.558993 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50262cd0-2091-4d2d-8757-1398fd50981b admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:51.563496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2e54f-7b60-4fa8-8989-f5e2588f26a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:51.564156 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2e54f-7b60-4fa8-8989-f5e2588f26a6 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:51.564410 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2e54f-7b60-4fa8-8989-f5e2588f26a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:51.564685 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2e54f-7b60-4fa8-8989-f5e2588f26a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:51.574870 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2e54f-7b60-4fa8-8989-f5e2588f26a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:51.576328 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2e54f-7b60-4fa8-8989-f5e2588f26a6 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:51.586567 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 910/1840] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:51.849025 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897d858-f95f-48b8-bbe5-d1bef19af404 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:51.850782 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-2897d858-f95f-48b8-bbe5-d1bef19af404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:51.866284 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897d858-f95f-48b8-bbe5-d1bef19af404 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:51.871311 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d338d3e-df1d-4855-84c5-fe94e6f4c79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:51.872090 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d338d3e-df1d-4855-84c5-fe94e6f4c79f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:51.872392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d338d3e-df1d-4855-84c5-fe94e6f4c79f admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:51.872674 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d338d3e-df1d-4855-84c5-fe94e6f4c79f admin admin] PATH_INFO: `/v3/projects/24e1f6a786ee43dead5baf7395538b88` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:51.873987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d338d3e-df1d-4855-84c5-fe94e6f4c79f admin admin] RBAC: Authorizing `identity:delete_project(project_id=24e1f6a786ee43dead5baf7395538b88)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:51.875957 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d338d3e-df1d-4855-84c5-fe94e6f4c79f admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:51.923319 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0d338d3e-df1d-4855-84c5-fe94e6f4c79f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24e1f6a786ee43dead5baf7395538b88', 'request_id': 'req-0d338d3e-df1d-4855-84c5-fe94e6f4c79f'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:51.929958 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0d338d3e-df1d-4855-84c5-fe94e6f4c79f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24e1f6a786ee43dead5baf7395538b88', 'request_id': 'req-0d338d3e-df1d-4855-84c5-fe94e6f4c79f'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:51.942076 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0d338d3e-df1d-4855-84c5-fe94e6f4c79f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24e1f6a786ee43dead5baf7395538b88', 'request_id': 'req-0d338d3e-df1d-4855-84c5-fe94e6f4c79f'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:51.948859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0d338d3e-df1d-4855-84c5-fe94e6f4c79f admin admin] The token cache is being invalidate because project 24e1f6a786ee43dead5baf7395538b88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:47:51.949264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0d338d3e-df1d-4855-84c5-fe94e6f4c79f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:51.949943 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0d338d3e-df1d-4855-84c5-fe94e6f4c79f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:51.951489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 931/1841] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:47:51 2022] DELETE /identity/v3/projects/24e1f6a786ee43dead5baf7395538b88 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:51.956648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf749f6-a59d-4e91-b83c-3ca50f7b6beb None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:51.957148 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf749f6-a59d-4e91-b83c-3ca50f7b6beb None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:51.957248 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf749f6-a59d-4e91-b83c-3ca50f7b6beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:51.975310 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-0bf749f6-a59d-4e91-b83c-3ca50f7b6beb None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:51.976409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-0bf749f6-a59d-4e91-b83c-3ca50f7b6beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:52.035641 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 911/1842] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:47:52.044262 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcaf64a-2e37-4cd0-a3d1-f8f857c873fa None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:52.044550 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcaf64a-2e37-4cd0-a3d1-f8f857c873fa None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:52.044854 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcaf64a-2e37-4cd0-a3d1-f8f857c873fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:52.063087 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-9fcaf64a-2e37-4cd0-a3d1-f8f857c873fa None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:52.064297 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-9fcaf64a-2e37-4cd0-a3d1-f8f857c873fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:52.073190 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 932/1843] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:52.077889 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf749f6-a59d-4e91-b83c-3ca50f7b6beb None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:52.087807 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf749f6-a59d-4e91-b83c-3ca50f7b6beb None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:52.093597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a0301-d150-47cd-ad86-fea6b56c0cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:52.094239 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a0301-d150-47cd-ad86-fea6b56c0cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:52.094515 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a0301-d150-47cd-ad86-fea6b56c0cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:52.094779 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a0301-d150-47cd-ad86-fea6b56c0cf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:52.095498 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a0301-d150-47cd-ad86-fea6b56c0cf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:47:52.095782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a0301-d150-47cd-ad86-fea6b56c0cf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:52.097413 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a0301-d150-47cd-ad86-fea6b56c0cf2 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:52.108882 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 912/1844] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:47:52.113233 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcaf64a-2e37-4cd0-a3d1-f8f857c873fa None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:52.122465 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcaf64a-2e37-4cd0-a3d1-f8f857c873fa None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:52.127543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e74d2-67df-4dac-a5a2-517021f27ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:52.128185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e74d2-67df-4dac-a5a2-517021f27ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:52.128462 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e74d2-67df-4dac-a5a2-517021f27ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:52.128775 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e74d2-67df-4dac-a5a2-517021f27ff6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:52.129473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e74d2-67df-4dac-a5a2-517021f27ff6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:52.131354 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e74d2-67df-4dac-a5a2-517021f27ff6 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:52.159544 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 933/1845] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:52 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:52.164872 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a0301-d150-47cd-ad86-fea6b56c0cf2 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:52.177035 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a0301-d150-47cd-ad86-fea6b56c0cf2 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:52.241878 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9f681-fca4-4575-9f70-01a36cce7d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:52.242549 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9f681-fca4-4575-9f70-01a36cce7d64 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:52.242828 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9f681-fca4-4575-9f70-01a36cce7d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:52.243103 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9f681-fca4-4575-9f70-01a36cce7d64 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:52.243785 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9f681-fca4-4575-9f70-01a36cce7d64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:52.245841 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9f681-fca4-4575-9f70-01a36cce7d64 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:52.270048 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 913/1846] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:52 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:52.271516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e74d2-67df-4dac-a5a2-517021f27ff6 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:52.282016 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e74d2-67df-4dac-a5a2-517021f27ff6 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:52.287198 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1a79a-68fd-49fb-8762-6d7713980269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:52.287869 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1a79a-68fd-49fb-8762-6d7713980269 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:52.288138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1a79a-68fd-49fb-8762-6d7713980269 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:52.288402 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1a79a-68fd-49fb-8762-6d7713980269 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:52.289159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd1a79a-68fd-49fb-8762-6d7713980269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:52.290800 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd1a79a-68fd-49fb-8762-6d7713980269 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:52.301501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 934/1847] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:52.305537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9f681-fca4-4575-9f70-01a36cce7d64 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:52.315504 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9f681-fca4-4575-9f70-01a36cce7d64 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:52.320247 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20133674-8875-49ac-a8c9-0d3ab4121b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:52.320936 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20133674-8875-49ac-a8c9-0d3ab4121b78 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:52.321214 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20133674-8875-49ac-a8c9-0d3ab4121b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:52.321483 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20133674-8875-49ac-a8c9-0d3ab4121b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:52.322248 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20133674-8875-49ac-a8c9-0d3ab4121b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:52.324384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20133674-8875-49ac-a8c9-0d3ab4121b78 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:52.337626 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 914/1848] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:52.338809 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1a79a-68fd-49fb-8762-6d7713980269 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:52.349118 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1a79a-68fd-49fb-8762-6d7713980269 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:52.353855 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03f7b0-88bd-498f-ba76-f2acde41b54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:52.354734 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03f7b0-88bd-498f-ba76-f2acde41b54b admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:52.355009 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03f7b0-88bd-498f-ba76-f2acde41b54b admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:52.355308 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03f7b0-88bd-498f-ba76-f2acde41b54b admin admin] PATH_INFO: `/v3/projects/c999d6422a4a462cb24fd4c18428109e/users/aab7a2c4346c43d2a56736d9d1a3371b/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:52.370115 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae03f7b0-88bd-498f-ba76-f2acde41b54b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c999d6422a4a462cb24fd4c18428109e, user_id=aab7a2c4346c43d2a56736d9d1a3371b, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:52.372945 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae03f7b0-88bd-498f-ba76-f2acde41b54b admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:52.386465 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 935/1849] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:52 2022] PUT /identity/v3/projects/c999d6422a4a462cb24fd4c18428109e/users/aab7a2c4346c43d2a56736d9d1a3371b/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:52.396755 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479e87a-b033-42d4-a4e3-b371baccad1e None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:52.397037 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479e87a-b033-42d4-a4e3-b371baccad1e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:52.397305 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479e87a-b033-42d4-a4e3-b371baccad1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:52.423429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-0479e87a-b033-42d4-a4e3-b371baccad1e None None] MFA Rules not processed for user `aab7a2c4346c43d2a56736d9d1a3371b`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:52.424620 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-0479e87a-b033-42d4-a4e3-b371baccad1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:52.487794 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 915/1850] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:52.492535 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03f7b0-88bd-498f-ba76-f2acde41b54b admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:52.554752 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03f7b0-88bd-498f-ba76-f2acde41b54b admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:52.559563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417683f3-d7ad-4eda-b1ba-75f875d4f6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:52.560218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417683f3-d7ad-4eda-b1ba-75f875d4f6f9 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:52.560495 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417683f3-d7ad-4eda-b1ba-75f875d4f6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:52.560802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417683f3-d7ad-4eda-b1ba-75f875d4f6f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:52.561539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417683f3-d7ad-4eda-b1ba-75f875d4f6f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:52.563454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417683f3-d7ad-4eda-b1ba-75f875d4f6f9 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:52.589137 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 936/1851] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:52 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:52.593432 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479e87a-b033-42d4-a4e3-b371baccad1e None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:52.603981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479e87a-b033-42d4-a4e3-b371baccad1e None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:52.660499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3fa387-da9f-453b-856b-6bebe89ce7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:52.661197 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fa387-da9f-453b-856b-6bebe89ce7c8 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:52.661477 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fa387-da9f-453b-856b-6bebe89ce7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:52.661742 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fa387-da9f-453b-856b-6bebe89ce7c8 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:52.662422 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3fa387-da9f-453b-856b-6bebe89ce7c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:52.664349 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3fa387-da9f-453b-856b-6bebe89ce7c8 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:52.686560 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 916/1852] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:52 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:52.690661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417683f3-d7ad-4eda-b1ba-75f875d4f6f9 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:52.700910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417683f3-d7ad-4eda-b1ba-75f875d4f6f9 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:52.705646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25de66-e058-44de-991f-9042288f797d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:52.706303 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25de66-e058-44de-991f-9042288f797d admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:52.706578 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25de66-e058-44de-991f-9042288f797d admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:52.706849 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25de66-e058-44de-991f-9042288f797d admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:52.707601 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b25de66-e058-44de-991f-9042288f797d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:52.709227 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b25de66-e058-44de-991f-9042288f797d admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:52.723427 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 937/1853] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:52.724874 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3fa387-da9f-453b-856b-6bebe89ce7c8 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:52.735015 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3fa387-da9f-453b-856b-6bebe89ce7c8 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:52.739902 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250fe37-44f6-48f9-bf23-2234173348f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:52.740799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250fe37-44f6-48f9-bf23-2234173348f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:52.741070 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250fe37-44f6-48f9-bf23-2234173348f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:52.741332 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250fe37-44f6-48f9-bf23-2234173348f0 admin admin] PATH_INFO: `/v3/projects/11ead7a7409846ebb364aea07f082a72/users/9534df0980d34f24a64c9c06952407c3/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:52.755777 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250fe37-44f6-48f9-bf23-2234173348f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11ead7a7409846ebb364aea07f082a72, user_id=9534df0980d34f24a64c9c06952407c3, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:52.758429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250fe37-44f6-48f9-bf23-2234173348f0 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:52.773223 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 917/1854] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:52 2022] PUT /identity/v3/projects/11ead7a7409846ebb364aea07f082a72/users/9534df0980d34f24a64c9c06952407c3/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:52.777129 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25de66-e058-44de-991f-9042288f797d admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:52.787995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25de66-e058-44de-991f-9042288f797d admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:52.844251 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f1ebe-0185-46b7-aaf7-630e0a694eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:52.844959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f1ebe-0185-46b7-aaf7-630e0a694eed admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:52.845265 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f1ebe-0185-46b7-aaf7-630e0a694eed admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:52.845532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f1ebe-0185-46b7-aaf7-630e0a694eed admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:52.846275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f1ebe-0185-46b7-aaf7-630e0a694eed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:52.847897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f1ebe-0185-46b7-aaf7-630e0a694eed admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:52.857641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 938/1855] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:52.862097 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250fe37-44f6-48f9-bf23-2234173348f0 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:52.872227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250fe37-44f6-48f9-bf23-2234173348f0 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:52.876950 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44069f1-4238-4b23-9164-3d9e51070bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:52.877848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44069f1-4238-4b23-9164-3d9e51070bcf admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:52.878119 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44069f1-4238-4b23-9164-3d9e51070bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:52.878384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44069f1-4238-4b23-9164-3d9e51070bcf admin admin] PATH_INFO: `/v3/projects/11ead7a7409846ebb364aea07f082a72/users/9534df0980d34f24a64c9c06952407c3/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:52.880373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44069f1-4238-4b23-9164-3d9e51070bcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=11ead7a7409846ebb364aea07f082a72, user_id=9534df0980d34f24a64c9c06952407c3, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:52.883036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44069f1-4238-4b23-9164-3d9e51070bcf admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:52.892595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 918/1856] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:52 2022] PUT /identity/v3/projects/11ead7a7409846ebb364aea07f082a72/users/9534df0980d34f24a64c9c06952407c3/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:47:52.903515 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2854a94-64fe-4b22-9001-498b7498ffba None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:52.903828 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2854a94-64fe-4b22-9001-498b7498ffba None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:52.904102 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2854a94-64fe-4b22-9001-498b7498ffba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:52.931641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-b2854a94-64fe-4b22-9001-498b7498ffba None None] MFA Rules not processed for user `9534df0980d34f24a64c9c06952407c3`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:52.932804 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b2854a94-64fe-4b22-9001-498b7498ffba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:53.002128 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 939/1857] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:52 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:47:53.036502 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44069f1-4238-4b23-9164-3d9e51070bcf admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:53.038273 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a44069f1-4238-4b23-9164-3d9e51070bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:53.091281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44069f1-4238-4b23-9164-3d9e51070bcf admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:53.095035 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f68dd-8e9e-4758-8b42-30162a2f8ade service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:53.095665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f68dd-8e9e-4758-8b42-30162a2f8ade service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:53.095910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f68dd-8e9e-4758-8b42-30162a2f8ade service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:53.096206 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f68dd-8e9e-4758-8b42-30162a2f8ade service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:53.154142 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f68dd-8e9e-4758-8b42-30162a2f8ade service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:53.155673 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f68dd-8e9e-4758-8b42-30162a2f8ade service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:53.165954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 919/1858] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:55.535163 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2854a94-64fe-4b22-9001-498b7498ffba None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:55.537064 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b2854a94-64fe-4b22-9001-498b7498ffba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:55.649849 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2854a94-64fe-4b22-9001-498b7498ffba None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:55.658822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04461e3a-6c68-4f25-afaf-f8e112c384f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:55.660143 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04461e3a-6c68-4f25-afaf-f8e112c384f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:55.660734 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04461e3a-6c68-4f25-afaf-f8e112c384f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:55.661162 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04461e3a-6c68-4f25-afaf-f8e112c384f0 admin admin] PATH_INFO: `/v3/users/87bad88fd9594220881c269df4490e61` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:55.662793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04461e3a-6c68-4f25-afaf-f8e112c384f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87bad88fd9594220881c269df4490e61)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:55.664913 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04461e3a-6c68-4f25-afaf-f8e112c384f0 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:55.752890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-04461e3a-6c68-4f25-afaf-f8e112c384f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87bad88fd9594220881c269df4490e61'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:55.764312 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-04461e3a-6c68-4f25-afaf-f8e112c384f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87bad88fd9594220881c269df4490e61'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:55.765384 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-04461e3a-6c68-4f25-afaf-f8e112c384f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87bad88fd9594220881c269df4490e61'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:55.777335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-04461e3a-6c68-4f25-afaf-f8e112c384f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87bad88fd9594220881c269df4490e61'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:55.778364 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-04461e3a-6c68-4f25-afaf-f8e112c384f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87bad88fd9594220881c269df4490e61'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:55.810214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 940/1859] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:47:55 2022] DELETE /identity/v3/users/87bad88fd9594220881c269df4490e61 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:55.815368 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5e6c8-77be-4efc-b27a-37e668628991 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:55.816497 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5e6c8-77be-4efc-b27a-37e668628991 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:55.817064 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5e6c8-77be-4efc-b27a-37e668628991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:55.844304 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-e6f5e6c8-77be-4efc-b27a-37e668628991 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:55.851631 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e6f5e6c8-77be-4efc-b27a-37e668628991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:55.963290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 920/1860] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:56.020775 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04461e3a-6c68-4f25-afaf-f8e112c384f0 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:56.023624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-04461e3a-6c68-4f25-afaf-f8e112c384f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:56.140005 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04461e3a-6c68-4f25-afaf-f8e112c384f0 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:56.145163 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185d3f8-dd2e-46c2-8001-e811443514b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:56.146653 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185d3f8-dd2e-46c2-8001-e811443514b0 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:56.147109 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185d3f8-dd2e-46c2-8001-e811443514b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:56.147917 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185d3f8-dd2e-46c2-8001-e811443514b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:56.161331 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a185d3f8-dd2e-46c2-8001-e811443514b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:56.162942 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a185d3f8-dd2e-46c2-8001-e811443514b0 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:56.176551 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 941/1861] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:56.890451 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5e6c8-77be-4efc-b27a-37e668628991 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:56.892444 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e6f5e6c8-77be-4efc-b27a-37e668628991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:56.906474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5e6c8-77be-4efc-b27a-37e668628991 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:56.911988 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1827b3-f402-4e3b-9a9a-df80ce059cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:56.913039 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1827b3-f402-4e3b-9a9a-df80ce059cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:56.913316 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1827b3-f402-4e3b-9a9a-df80ce059cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:56.913587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1827b3-f402-4e3b-9a9a-df80ce059cd8 admin admin] PATH_INFO: `/v3/users/aab7a2c4346c43d2a56736d9d1a3371b` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:56.914784 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1827b3-f402-4e3b-9a9a-df80ce059cd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aab7a2c4346c43d2a56736d9d1a3371b)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:56.916715 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1827b3-f402-4e3b-9a9a-df80ce059cd8 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:56.985378 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0b1827b3-f402-4e3b-9a9a-df80ce059cd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aab7a2c4346c43d2a56736d9d1a3371b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:56.992649 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0b1827b3-f402-4e3b-9a9a-df80ce059cd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aab7a2c4346c43d2a56736d9d1a3371b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:57.006208 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0b1827b3-f402-4e3b-9a9a-df80ce059cd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aab7a2c4346c43d2a56736d9d1a3371b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:57.006896 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0b1827b3-f402-4e3b-9a9a-df80ce059cd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aab7a2c4346c43d2a56736d9d1a3371b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:57.011215 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0b1827b3-f402-4e3b-9a9a-df80ce059cd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aab7a2c4346c43d2a56736d9d1a3371b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:57.057928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 921/1862] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:47:56 2022] DELETE /identity/v3/users/aab7a2c4346c43d2a56736d9d1a3371b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:57.059884 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227526f2-3395-4efe-9faa-f517960e535c None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:57.060329 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227526f2-3395-4efe-9faa-f517960e535c None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:57.066342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227526f2-3395-4efe-9faa-f517960e535c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:57.090189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-227526f2-3395-4efe-9faa-f517960e535c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:57.091855 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-227526f2-3395-4efe-9faa-f517960e535c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:57.206063 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 942/1863] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:57.227409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1827b3-f402-4e3b-9a9a-df80ce059cd8 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:57.229259 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-0b1827b3-f402-4e3b-9a9a-df80ce059cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:57.329539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1827b3-f402-4e3b-9a9a-df80ce059cd8 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:57.338685 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11462f-2dd8-42fc-9726-f975eb7318ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:57.341775 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11462f-2dd8-42fc-9726-f975eb7318ee service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:57.342069 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11462f-2dd8-42fc-9726-f975eb7318ee service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:57.342381 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11462f-2dd8-42fc-9726-f975eb7318ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:57.378851 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11462f-2dd8-42fc-9726-f975eb7318ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:57.380392 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11462f-2dd8-42fc-9726-f975eb7318ee service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:57.392156 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 922/1864] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:57.792679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227526f2-3395-4efe-9faa-f517960e535c None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:57.794169 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-227526f2-3395-4efe-9faa-f517960e535c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:57.817765 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227526f2-3395-4efe-9faa-f517960e535c None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:57.839292 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389613f-237c-441d-a7bf-568416303de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:57.839292 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389613f-237c-441d-a7bf-568416303de0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:57.839292 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389613f-237c-441d-a7bf-568416303de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:57.839292 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389613f-237c-441d-a7bf-568416303de0 admin admin] PATH_INFO: `/v3/projects/1f6b9be562344f37b2318ef3201e10c1` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:57.839665 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389613f-237c-441d-a7bf-568416303de0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f6b9be562344f37b2318ef3201e10c1)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:57.840928 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389613f-237c-441d-a7bf-568416303de0 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:57.924776 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11462f-2dd8-42fc-9726-f975eb7318ee service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:57.926921 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-4a11462f-2dd8-42fc-9726-f975eb7318ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:57.931561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0389613f-237c-441d-a7bf-568416303de0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f6b9be562344f37b2318ef3201e10c1', 'request_id': 'req-0389613f-237c-441d-a7bf-568416303de0'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:57.940320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0389613f-237c-441d-a7bf-568416303de0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f6b9be562344f37b2318ef3201e10c1', 'request_id': 'req-0389613f-237c-441d-a7bf-568416303de0'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:57.947535 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0389613f-237c-441d-a7bf-568416303de0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f6b9be562344f37b2318ef3201e10c1', 'request_id': 'req-0389613f-237c-441d-a7bf-568416303de0'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:57.949316 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11462f-2dd8-42fc-9726-f975eb7318ee service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:57.989776 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0389613f-237c-441d-a7bf-568416303de0 admin admin] The token cache is being invalidate because project 1f6b9be562344f37b2318ef3201e10c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:47:57.990355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0389613f-237c-441d-a7bf-568416303de0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:57.991056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0389613f-237c-441d-a7bf-568416303de0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:57.992740 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 943/1865] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:47:57 2022] DELETE /identity/v3/projects/1f6b9be562344f37b2318ef3201e10c1 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:57.996555 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389613f-237c-441d-a7bf-568416303de0 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:57.999389 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-0389613f-237c-441d-a7bf-568416303de0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:58.046259 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40278a-6b1c-4cf6-9909-c4181215ffa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:58.047427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40278a-6b1c-4cf6-9909-c4181215ffa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:58.047860 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40278a-6b1c-4cf6-9909-c4181215ffa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:58.048276 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40278a-6b1c-4cf6-9909-c4181215ffa8 admin admin] PATH_INFO: `/v3/projects/c999d6422a4a462cb24fd4c18428109e` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:58.049950 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40278a-6b1c-4cf6-9909-c4181215ffa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c999d6422a4a462cb24fd4c18428109e)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:58.053692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40278a-6b1c-4cf6-9909-c4181215ffa8 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:58.080614 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389613f-237c-441d-a7bf-568416303de0 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:58.086332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e49a9-4f59-40c1-9509-87cddac5bd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:58.087789 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e49a9-4f59-40c1-9509-87cddac5bd34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:58.088244 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e49a9-4f59-40c1-9509-87cddac5bd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:58.088516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e49a9-4f59-40c1-9509-87cddac5bd34 admin admin] PATH_INFO: `/v3/users/805b6824405444dc807de044dafe4452` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:58.089902 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e49a9-4f59-40c1-9509-87cddac5bd34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=805b6824405444dc807de044dafe4452)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:58.091897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e49a9-4f59-40c1-9509-87cddac5bd34 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:58.112885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bf40278a-6b1c-4cf6-9909-c4181215ffa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c999d6422a4a462cb24fd4c18428109e', 'request_id': 'req-bf40278a-6b1c-4cf6-9909-c4181215ffa8'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:58.121209 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bf40278a-6b1c-4cf6-9909-c4181215ffa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c999d6422a4a462cb24fd4c18428109e', 'request_id': 'req-bf40278a-6b1c-4cf6-9909-c4181215ffa8'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:58.129475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bf40278a-6b1c-4cf6-9909-c4181215ffa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c999d6422a4a462cb24fd4c18428109e', 'request_id': 'req-bf40278a-6b1c-4cf6-9909-c4181215ffa8'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:58.144259 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bf40278a-6b1c-4cf6-9909-c4181215ffa8 admin admin] The token cache is being invalidate because project c999d6422a4a462cb24fd4c18428109e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:47:58.145417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bf40278a-6b1c-4cf6-9909-c4181215ffa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:58.146395 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bf40278a-6b1c-4cf6-9909-c4181215ffa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:58.148507 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 923/1866] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:47:57 2022] DELETE /identity/v3/projects/c999d6422a4a462cb24fd4c18428109e => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:58.148788 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-079e49a9-4f59-40c1-9509-87cddac5bd34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '805b6824405444dc807de044dafe4452'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:58.152118 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40278a-6b1c-4cf6-9909-c4181215ffa8 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:58.153940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-bf40278a-6b1c-4cf6-9909-c4181215ffa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:58.159912 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-079e49a9-4f59-40c1-9509-87cddac5bd34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '805b6824405444dc807de044dafe4452'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:58.160571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-079e49a9-4f59-40c1-9509-87cddac5bd34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '805b6824405444dc807de044dafe4452'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:58.174550 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-079e49a9-4f59-40c1-9509-87cddac5bd34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '805b6824405444dc807de044dafe4452'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:58.175230 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-079e49a9-4f59-40c1-9509-87cddac5bd34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '805b6824405444dc807de044dafe4452'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:58.204913 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 944/1867] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:47:57 2022] DELETE /identity/v3/users/805b6824405444dc807de044dafe4452 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:47:58.239315 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40278a-6b1c-4cf6-9909-c4181215ffa8 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:58.299678 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3daef-2cae-441c-a6c1-b6d6e9319e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:58.300936 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3daef-2cae-441c-a6c1-b6d6e9319e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:58.301563 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3daef-2cae-441c-a6c1-b6d6e9319e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:58.301852 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3daef-2cae-441c-a6c1-b6d6e9319e68 admin admin] PATH_INFO: `/v3/users/9534df0980d34f24a64c9c06952407c3` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:58.305504 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3daef-2cae-441c-a6c1-b6d6e9319e68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9534df0980d34f24a64c9c06952407c3)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:58.307989 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3daef-2cae-441c-a6c1-b6d6e9319e68 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:58.380265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-7cf3daef-2cae-441c-a6c1-b6d6e9319e68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9534df0980d34f24a64c9c06952407c3'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:58.390084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-7cf3daef-2cae-441c-a6c1-b6d6e9319e68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9534df0980d34f24a64c9c06952407c3'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:58.402249 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-7cf3daef-2cae-441c-a6c1-b6d6e9319e68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9534df0980d34f24a64c9c06952407c3'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:58.403125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-7cf3daef-2cae-441c-a6c1-b6d6e9319e68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9534df0980d34f24a64c9c06952407c3'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:58.403949 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-7cf3daef-2cae-441c-a6c1-b6d6e9319e68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9534df0980d34f24a64c9c06952407c3'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:58.425018 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 924/1868] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:47:58 2022] DELETE /identity/v3/users/9534df0980d34f24a64c9c06952407c3 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:58.429756 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b2298-7695-4423-a86d-38d18ea6c004 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:58.430085 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b2298-7695-4423-a86d-38d18ea6c004 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:58.430332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b2298-7695-4423-a86d-38d18ea6c004 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:58.432274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 945/1869] 10.222.0.26 () {54 vars in 921 bytes} [Mon May 9 09:47:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:47:58.448361 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445a6d0-8c4b-49fe-8140-1599643f0af9 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:58.450831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445a6d0-8c4b-49fe-8140-1599643f0af9 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:58.451970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445a6d0-8c4b-49fe-8140-1599643f0af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:58.486834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-0445a6d0-8c4b-49fe-8140-1599643f0af9 None None] MFA Rules not processed for user `6362303cb918486c8525225914332ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:58.489861 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-0445a6d0-8c4b-49fe-8140-1599643f0af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:58.584028 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 925/1870] 10.222.0.26 () {58 vars in 1018 bytes} [Mon May 9 09:47:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:58.597270 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b2298-7695-4423-a86d-38d18ea6c004 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:58.598986 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b79b2298-7695-4423-a86d-38d18ea6c004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:58.624300 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b2298-7695-4423-a86d-38d18ea6c004 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:58.638563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff00b1-87bd-4d51-8951-941165806d53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:58.639482 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff00b1-87bd-4d51-8951-941165806d53 service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:58.639829 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff00b1-87bd-4d51-8951-941165806d53 service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:58.640343 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff00b1-87bd-4d51-8951-941165806d53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:58.658294 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff00b1-87bd-4d51-8951-941165806d53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:58.660525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff00b1-87bd-4d51-8951-941165806d53 service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:58.674018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 946/1871] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:58.679712 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445a6d0-8c4b-49fe-8140-1599643f0af9 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:58.681320 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-0445a6d0-8c4b-49fe-8140-1599643f0af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:58.760175 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445a6d0-8c4b-49fe-8140-1599643f0af9 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:58.766204 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8bd54-9dd0-4869-bb76-4e64c17baa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:58.767137 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8bd54-9dd0-4869-bb76-4e64c17baa6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:58.767519 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8bd54-9dd0-4869-bb76-4e64c17baa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:58.767701 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8bd54-9dd0-4869-bb76-4e64c17baa6d admin admin] PATH_INFO: `/v3/projects/274372d2bda549dea5145b84ead70ee8` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:58.769514 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8bd54-9dd0-4869-bb76-4e64c17baa6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=274372d2bda549dea5145b84ead70ee8)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:58.771605 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8bd54-9dd0-4869-bb76-4e64c17baa6d admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:58.831527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff00b1-87bd-4d51-8951-941165806d53 service nova] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:58.833103 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-68ff00b1-87bd-4d51-8951-941165806d53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:58.848318 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4aa8bd54-9dd0-4869-bb76-4e64c17baa6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '274372d2bda549dea5145b84ead70ee8', 'request_id': 'req-4aa8bd54-9dd0-4869-bb76-4e64c17baa6d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:58.856125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4aa8bd54-9dd0-4869-bb76-4e64c17baa6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '274372d2bda549dea5145b84ead70ee8', 'request_id': 'req-4aa8bd54-9dd0-4869-bb76-4e64c17baa6d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:58.856284 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff00b1-87bd-4d51-8951-941165806d53 service nova] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:58.868799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4aa8bd54-9dd0-4869-bb76-4e64c17baa6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '274372d2bda549dea5145b84ead70ee8', 'request_id': 'req-4aa8bd54-9dd0-4869-bb76-4e64c17baa6d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:58.899975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4aa8bd54-9dd0-4869-bb76-4e64c17baa6d admin admin] The token cache is being invalidate because project 274372d2bda549dea5145b84ead70ee8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:47:58.901253 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4aa8bd54-9dd0-4869-bb76-4e64c17baa6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:58.901952 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4aa8bd54-9dd0-4869-bb76-4e64c17baa6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:58.903847 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 926/1872] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:47:58 2022] DELETE /identity/v3/projects/274372d2bda549dea5145b84ead70ee8 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:58.908436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da860ed-5689-48f3-8515-e9c3e3abbfd8 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:58.908756 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da860ed-5689-48f3-8515-e9c3e3abbfd8 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:58.909031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da860ed-5689-48f3-8515-e9c3e3abbfd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:58.931399 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-0da860ed-5689-48f3-8515-e9c3e3abbfd8 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:58.932604 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-0da860ed-5689-48f3-8515-e9c3e3abbfd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:58.940271 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb7047-f840-4a4b-968d-7b3556bb3253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:58.941380 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb7047-f840-4a4b-968d-7b3556bb3253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:58.941574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb7047-f840-4a4b-968d-7b3556bb3253 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:58.941892 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb7047-f840-4a4b-968d-7b3556bb3253 admin admin] PATH_INFO: `/v3/projects/11ead7a7409846ebb364aea07f082a72` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:58.943117 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb7047-f840-4a4b-968d-7b3556bb3253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11ead7a7409846ebb364aea07f082a72)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:58.945136 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb7047-f840-4a4b-968d-7b3556bb3253 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:59.002302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 927/1873] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:59.014938 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-82fb7047-f840-4a4b-968d-7b3556bb3253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11ead7a7409846ebb364aea07f082a72', 'request_id': 'req-82fb7047-f840-4a4b-968d-7b3556bb3253'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:59.016118 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aba319-4dcc-44d4-ac81-6ecbd748579b None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:59.016403 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aba319-4dcc-44d4-ac81-6ecbd748579b None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:59.021525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aba319-4dcc-44d4-ac81-6ecbd748579b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:59.021770 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-82fb7047-f840-4a4b-968d-7b3556bb3253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11ead7a7409846ebb364aea07f082a72', 'request_id': 'req-82fb7047-f840-4a4b-968d-7b3556bb3253'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:59.030384 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-82fb7047-f840-4a4b-968d-7b3556bb3253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11ead7a7409846ebb364aea07f082a72', 'request_id': 'req-82fb7047-f840-4a4b-968d-7b3556bb3253'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:59.043683 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-74aba319-4dcc-44d4-ac81-6ecbd748579b None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:59.045064 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-74aba319-4dcc-44d4-ac81-6ecbd748579b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:59.069256 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-82fb7047-f840-4a4b-968d-7b3556bb3253 admin admin] The token cache is being invalidate because project 11ead7a7409846ebb364aea07f082a72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:47:59.069551 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-82fb7047-f840-4a4b-968d-7b3556bb3253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:59.070289 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-82fb7047-f840-4a4b-968d-7b3556bb3253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:47:59.072165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 947/1874] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:47:58 2022] DELETE /identity/v3/projects/11ead7a7409846ebb364aea07f082a72 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:47:59.078750 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0910a8-e041-4af6-a3ad-bc7732271dbf None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:59.079113 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0910a8-e041-4af6-a3ad-bc7732271dbf None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:59.079354 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0910a8-e041-4af6-a3ad-bc7732271dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:59.099010 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-cd0910a8-e041-4af6-a3ad-bc7732271dbf None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:59.100964 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-cd0910a8-e041-4af6-a3ad-bc7732271dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:59.125260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 928/1875] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:59.130041 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aba319-4dcc-44d4-ac81-6ecbd748579b None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:59.142752 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aba319-4dcc-44d4-ac81-6ecbd748579b None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:59.149649 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a0c12-6015-4981-a76d-ac1b91621bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:59.150544 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a0c12-6015-4981-a76d-ac1b91621bfc admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:59.150914 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a0c12-6015-4981-a76d-ac1b91621bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:59.151273 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a0c12-6015-4981-a76d-ac1b91621bfc admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:59.152389 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a0c12-6015-4981-a76d-ac1b91621bfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:47:59.152879 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a0c12-6015-4981-a76d-ac1b91621bfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:59.157979 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a0c12-6015-4981-a76d-ac1b91621bfc admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:59.171962 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 929/1876] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:47:59.184645 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 948/1877] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:47:59.185292 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a0c12-6015-4981-a76d-ac1b91621bfc admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:59.192200 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be397517-5ffe-4307-b248-8610e3f15869 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:59.192488 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be397517-5ffe-4307-b248-8610e3f15869 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:59.192821 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be397517-5ffe-4307-b248-8610e3f15869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:59.204687 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a0c12-6015-4981-a76d-ac1b91621bfc admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:59.213943 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-be397517-5ffe-4307-b248-8610e3f15869 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:59.214429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce6712-2380-44fc-b6d8-28ce253eaff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:59.215185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce6712-2380-44fc-b6d8-28ce253eaff5 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:59.215298 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-be397517-5ffe-4307-b248-8610e3f15869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:59.216554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce6712-2380-44fc-b6d8-28ce253eaff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:59.218230 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce6712-2380-44fc-b6d8-28ce253eaff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:59.219037 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce6712-2380-44fc-b6d8-28ce253eaff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:59.224163 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce6712-2380-44fc-b6d8-28ce253eaff5 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:59.225394 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 949/1878] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:47:59.228377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be397517-5ffe-4307-b248-8610e3f15869 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:59.243434 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be397517-5ffe-4307-b248-8610e3f15869 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:59.260211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d51c3-81ae-437d-b880-ccca51ad8c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:59.261256 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d51c3-81ae-437d-b880-ccca51ad8c81 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:59.261570 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d51c3-81ae-437d-b880-ccca51ad8c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:59.261799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d51c3-81ae-437d-b880-ccca51ad8c81 admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:59.263207 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d51c3-81ae-437d-b880-ccca51ad8c81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:47:59.263502 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d51c3-81ae-437d-b880-ccca51ad8c81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:59.267898 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d51c3-81ae-437d-b880-ccca51ad8c81 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:59.288025 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 950/1879] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:47:59.288594 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 930/1880] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:59 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:59.292959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d51c3-81ae-437d-b880-ccca51ad8c81 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:59.294845 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce6712-2380-44fc-b6d8-28ce253eaff5 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:59.325819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d51c3-81ae-437d-b880-ccca51ad8c81 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:59.384712 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce6712-2380-44fc-b6d8-28ce253eaff5 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:59.391026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc37f66-4320-477e-ae12-756d2f1dfd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:59.392450 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc37f66-4320-477e-ae12-756d2f1dfd7f admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:59.392822 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc37f66-4320-477e-ae12-756d2f1dfd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:59.393460 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc37f66-4320-477e-ae12-756d2f1dfd7f admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:59.394387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc37f66-4320-477e-ae12-756d2f1dfd7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:59.397587 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925ec43-d684-4384-a1d2-d7ca178c07d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:59.397736 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc37f66-4320-477e-ae12-756d2f1dfd7f admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:59.398380 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925ec43-d684-4384-a1d2-d7ca178c07d6 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:59.398733 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925ec43-d684-4384-a1d2-d7ca178c07d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:59.399073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925ec43-d684-4384-a1d2-d7ca178c07d6 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:59.399832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925ec43-d684-4384-a1d2-d7ca178c07d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:59.402075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925ec43-d684-4384-a1d2-d7ca178c07d6 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:59.431834 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 951/1881] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:59 2022] POST /identity/v3/users => generated 291 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:59.432967 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925ec43-d684-4384-a1d2-d7ca178c07d6 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:59.436381 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 931/1882] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:59 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:47:59.437484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc37f66-4320-477e-ae12-756d2f1dfd7f admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:59.446124 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925ec43-d684-4384-a1d2-d7ca178c07d6 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:59.510858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d884c-b6f1-4865-a978-039e7cf0a39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:59.511541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d884c-b6f1-4865-a978-039e7cf0a39f admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:59.511827 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d884c-b6f1-4865-a978-039e7cf0a39f admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:59.512104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d884c-b6f1-4865-a978-039e7cf0a39f admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:59.512871 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650d884c-b6f1-4865-a978-039e7cf0a39f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:59.514552 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650d884c-b6f1-4865-a978-039e7cf0a39f admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:59.516591 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc37f66-4320-477e-ae12-756d2f1dfd7f admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:59.521453 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b60798-44f7-401c-921d-191fcbfa8969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:59.522172 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b60798-44f7-401c-921d-191fcbfa8969 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:59.522521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b60798-44f7-401c-921d-191fcbfa8969 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:59.523186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b60798-44f7-401c-921d-191fcbfa8969 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:59.523622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b60798-44f7-401c-921d-191fcbfa8969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:59.525642 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b60798-44f7-401c-921d-191fcbfa8969 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:59.528713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 952/1883] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:47:59.529912 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d884c-b6f1-4865-a978-039e7cf0a39f admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:59.543005 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d884c-b6f1-4865-a978-039e7cf0a39f admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:59.547795 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9779f-a8ad-43ea-9b9c-5f2ebddd6b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:59.548114 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 932/1884] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:59 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:59.548488 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9779f-a8ad-43ea-9b9c-5f2ebddd6b7c admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:59.548798 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9779f-a8ad-43ea-9b9c-5f2ebddd6b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:59.549069 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9779f-a8ad-43ea-9b9c-5f2ebddd6b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:59.549796 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada9779f-a8ad-43ea-9b9c-5f2ebddd6b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:59.551401 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada9779f-a8ad-43ea-9b9c-5f2ebddd6b7c admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:59.552090 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b60798-44f7-401c-921d-191fcbfa8969 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:59.561777 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 953/1885] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:47:59.565576 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9779f-a8ad-43ea-9b9c-5f2ebddd6b7c admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:59.565780 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b60798-44f7-401c-921d-191fcbfa8969 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:59.570458 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c811ada-6190-4bc1-aada-28a433bbabb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:59.571130 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c811ada-6190-4bc1-aada-28a433bbabb1 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:59.571412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c811ada-6190-4bc1-aada-28a433bbabb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:59.571684 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c811ada-6190-4bc1-aada-28a433bbabb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:59.572413 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c811ada-6190-4bc1-aada-28a433bbabb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:59.574091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c811ada-6190-4bc1-aada-28a433bbabb1 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:59.575069 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9779f-a8ad-43ea-9b9c-5f2ebddd6b7c admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:59.579737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e2c74-f37e-43de-b3c3-4f728868229b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:59.580606 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e2c74-f37e-43de-b3c3-4f728868229b admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:59.580916 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e2c74-f37e-43de-b3c3-4f728868229b admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:59.581195 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e2c74-f37e-43de-b3c3-4f728868229b admin admin] PATH_INFO: `/v3/projects/28f8c73c343642b099112937878eea43/users/20daa702c71b4d29b0b7d42fb12a3d6c/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:59.584790 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 933/1886] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:47:59.588096 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c811ada-6190-4bc1-aada-28a433bbabb1 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:59.595821 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e2c74-f37e-43de-b3c3-4f728868229b admin admin] RBAC: Authorizing `identity:create_grant(project_id=28f8c73c343642b099112937878eea43, user_id=20daa702c71b4d29b0b7d42fb12a3d6c, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:59.597707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c811ada-6190-4bc1-aada-28a433bbabb1 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:59.598563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e2c74-f37e-43de-b3c3-4f728868229b admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:59.602462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13156d2f-5a10-4026-bce4-688b9c9c6334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:59.603011 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13156d2f-5a10-4026-bce4-688b9c9c6334 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:59.603295 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13156d2f-5a10-4026-bce4-688b9c9c6334 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:59.603566 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13156d2f-5a10-4026-bce4-688b9c9c6334 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:59.604283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13156d2f-5a10-4026-bce4-688b9c9c6334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:59.605945 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13156d2f-5a10-4026-bce4-688b9c9c6334 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:59.612366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 954/1887] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:59 2022] PUT /identity/v3/projects/28f8c73c343642b099112937878eea43/users/20daa702c71b4d29b0b7d42fb12a3d6c/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:47:59.619975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 934/1888] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:47:59.621458 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e2c74-f37e-43de-b3c3-4f728868229b admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:59.622361 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975121b-dbec-4552-8390-9e85f3ed9f28 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:59.622737 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975121b-dbec-4552-8390-9e85f3ed9f28 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:59.623078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975121b-dbec-4552-8390-9e85f3ed9f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:59.649542 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-4975121b-dbec-4552-8390-9e85f3ed9f28 None None] MFA Rules not processed for user `20daa702c71b4d29b0b7d42fb12a3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:59.650823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-4975121b-dbec-4552-8390-9e85f3ed9f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:59.678738 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e2c74-f37e-43de-b3c3-4f728868229b admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:59.683339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c688a9-bc62-40a6-b905-c0e8d34a2cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:59.684155 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c688a9-bc62-40a6-b905-c0e8d34a2cfb admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:59.684432 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c688a9-bc62-40a6-b905-c0e8d34a2cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:59.684721 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c688a9-bc62-40a6-b905-c0e8d34a2cfb admin admin] PATH_INFO: `/v3/projects/39c36cb22c2846e480c26840635bf29c/users/1c091a71e2ad46ceafe25be874c016a6/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:59.698652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c688a9-bc62-40a6-b905-c0e8d34a2cfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=39c36cb22c2846e480c26840635bf29c, user_id=1c091a71e2ad46ceafe25be874c016a6, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:59.701303 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c688a9-bc62-40a6-b905-c0e8d34a2cfb admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:59.713616 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 955/1889] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:59 2022] PUT /identity/v3/projects/39c36cb22c2846e480c26840635bf29c/users/1c091a71e2ad46ceafe25be874c016a6/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:47:59.718629 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d88d01-64ff-4fd0-847a-699c944624f8 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:59.719022 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d88d01-64ff-4fd0-847a-699c944624f8 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:59.719354 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d88d01-64ff-4fd0-847a-699c944624f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:59.720702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 935/1890] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:47:59.724954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975121b-dbec-4552-8390-9e85f3ed9f28 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:59.734727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975121b-dbec-4552-8390-9e85f3ed9f28 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:59.737389 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-c0d88d01-64ff-4fd0-847a-699c944624f8 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:47:59.738589 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c0d88d01-64ff-4fd0-847a-699c944624f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:59.789674 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046ed07-ff56-483a-b2d6-bdd73dd801bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:59.790476 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046ed07-ff56-483a-b2d6-bdd73dd801bf admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:59.790755 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046ed07-ff56-483a-b2d6-bdd73dd801bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:59.791024 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046ed07-ff56-483a-b2d6-bdd73dd801bf admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:59.791719 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5046ed07-ff56-483a-b2d6-bdd73dd801bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:59.793651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5046ed07-ff56-483a-b2d6-bdd73dd801bf admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:59.800273 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 956/1891] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:47:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:47:59.811201 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d88d01-64ff-4fd0-847a-699c944624f8 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:59.813159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c0d88d01-64ff-4fd0-847a-699c944624f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:47:59.822762 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 936/1892] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:47:59 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:47:59.823915 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046ed07-ff56-483a-b2d6-bdd73dd801bf admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:59.870294 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d88d01-64ff-4fd0-847a-699c944624f8 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:59.889008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046ed07-ff56-483a-b2d6-bdd73dd801bf admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:59.893932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6024307-ca1f-4d19-aacb-a7a5fcc24c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:59.894603 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6024307-ca1f-4d19-aacb-a7a5fcc24c30 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:59.894882 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6024307-ca1f-4d19-aacb-a7a5fcc24c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:59.895150 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6024307-ca1f-4d19-aacb-a7a5fcc24c30 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:59.895843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6024307-ca1f-4d19-aacb-a7a5fcc24c30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:59.897804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6024307-ca1f-4d19-aacb-a7a5fcc24c30 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:59.918886 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 937/1893] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:47:59 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:47:59.926517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6024307-ca1f-4d19-aacb-a7a5fcc24c30 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:59.926747 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10fc9c-80f5-4b14-99d7-a33a22af59b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:59.926747 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10fc9c-80f5-4b14-99d7-a33a22af59b3 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:59.926747 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10fc9c-80f5-4b14-99d7-a33a22af59b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:59.926747 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10fc9c-80f5-4b14-99d7-a33a22af59b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:59.934848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10fc9c-80f5-4b14-99d7-a33a22af59b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:59.936357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10fc9c-80f5-4b14-99d7-a33a22af59b3 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:59.936454 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6024307-ca1f-4d19-aacb-a7a5fcc24c30 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:59.942032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f54caf5-66de-437a-931c-7e7233fa1c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:59.942700 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f54caf5-66de-437a-931c-7e7233fa1c74 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:59.942974 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f54caf5-66de-437a-931c-7e7233fa1c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:59.943353 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f54caf5-66de-437a-931c-7e7233fa1c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:59.944098 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f54caf5-66de-437a-931c-7e7233fa1c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:59.945786 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f54caf5-66de-437a-931c-7e7233fa1c74 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:47:59.946175 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 957/1894] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:47:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:47:59.961455 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 938/1895] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:47:59.963113 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10fc9c-80f5-4b14-99d7-a33a22af59b3 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:47:59.972786 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10fc9c-80f5-4b14-99d7-a33a22af59b3 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:47:59.977802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff15bc4-aa32-4f7f-9183-0b9ff8042184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:47:59.978684 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff15bc4-aa32-4f7f-9183-0b9ff8042184 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:47:59.978960 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff15bc4-aa32-4f7f-9183-0b9ff8042184 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:47:59.979230 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff15bc4-aa32-4f7f-9183-0b9ff8042184 admin admin] PATH_INFO: `/v3/projects/5086057dd20d41bb8e2645ada136c422/users/4f15ef0f7b374345a0f82e3c36bce49c/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:47:59.993814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff15bc4-aa32-4f7f-9183-0b9ff8042184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5086057dd20d41bb8e2645ada136c422, user_id=4f15ef0f7b374345a0f82e3c36bce49c, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:47:59.996517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff15bc4-aa32-4f7f-9183-0b9ff8042184 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:00.010157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 958/1896] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:47:59 2022] PUT /identity/v3/projects/5086057dd20d41bb8e2645ada136c422/users/4f15ef0f7b374345a0f82e3c36bce49c/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:48:00.014247 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f54caf5-66de-437a-931c-7e7233fa1c74 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:00.024382 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f54caf5-66de-437a-931c-7e7233fa1c74 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:00.083424 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2b6e5-e285-4277-b659-38ec9febf691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:00.084104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2b6e5-e285-4277-b659-38ec9febf691 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:00.084384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2b6e5-e285-4277-b659-38ec9febf691 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:00.084655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2b6e5-e285-4277-b659-38ec9febf691 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:00.085410 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc2b6e5-e285-4277-b659-38ec9febf691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:00.087177 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc2b6e5-e285-4277-b659-38ec9febf691 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:00.103742 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 939/1897] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:00.106983 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff15bc4-aa32-4f7f-9183-0b9ff8042184 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:00.118679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff15bc4-aa32-4f7f-9183-0b9ff8042184 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:00.124795 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629d861-6f4b-4282-a132-d39b1e2db40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:00.125761 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629d861-6f4b-4282-a132-d39b1e2db40e admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:00.126017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629d861-6f4b-4282-a132-d39b1e2db40e admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:00.126283 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629d861-6f4b-4282-a132-d39b1e2db40e admin admin] PATH_INFO: `/v3/projects/5086057dd20d41bb8e2645ada136c422/users/4f15ef0f7b374345a0f82e3c36bce49c/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:00.128345 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9629d861-6f4b-4282-a132-d39b1e2db40e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5086057dd20d41bb8e2645ada136c422, user_id=4f15ef0f7b374345a0f82e3c36bce49c, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:00.131883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9629d861-6f4b-4282-a132-d39b1e2db40e admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:00.146705 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 959/1898] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:00 2022] PUT /identity/v3/projects/5086057dd20d41bb8e2645ada136c422/users/4f15ef0f7b374345a0f82e3c36bce49c/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:48:00.158520 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a9d34-a71b-4d3e-aa77-d85e7dae0c4f None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:00.158894 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a9d34-a71b-4d3e-aa77-d85e7dae0c4f None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:00.159230 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a9d34-a71b-4d3e-aa77-d85e7dae0c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:00.190591 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-524a9d34-a71b-4d3e-aa77-d85e7dae0c4f None None] MFA Rules not processed for user `4f15ef0f7b374345a0f82e3c36bce49c`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:00.191830 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-524a9d34-a71b-4d3e-aa77-d85e7dae0c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:00.263851 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 940/1899] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:00 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:48:00.271672 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629d861-6f4b-4282-a132-d39b1e2db40e admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:00.274333 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-9629d861-6f4b-4282-a132-d39b1e2db40e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:00.349056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629d861-6f4b-4282-a132-d39b1e2db40e admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:00.356944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ce6e0-6d0a-40db-93f4-1a45b35efddd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:00.357676 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ce6e0-6d0a-40db-93f4-1a45b35efddd service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:00.358086 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ce6e0-6d0a-40db-93f4-1a45b35efddd service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:00.358453 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ce6e0-6d0a-40db-93f4-1a45b35efddd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:00.371208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ce6e0-6d0a-40db-93f4-1a45b35efddd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:00.372988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ce6e0-6d0a-40db-93f4-1a45b35efddd service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:00.390685 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 960/1900] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:00 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:48:00.891000 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a9d34-a71b-4d3e-aa77-d85e7dae0c4f None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:00.972020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a9d34-a71b-4d3e-aa77-d85e7dae0c4f None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:00.978732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377fa180-73b1-428f-8f57-fa5d4e66da99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:00.979619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fa180-73b1-428f-8f57-fa5d4e66da99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:00.979890 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fa180-73b1-428f-8f57-fa5d4e66da99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:00.980155 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fa180-73b1-428f-8f57-fa5d4e66da99 admin admin] PATH_INFO: `/v3/users/20daa702c71b4d29b0b7d42fb12a3d6c` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:00.981405 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377fa180-73b1-428f-8f57-fa5d4e66da99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20daa702c71b4d29b0b7d42fb12a3d6c)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:00.984063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377fa180-73b1-428f-8f57-fa5d4e66da99 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:01.037682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-377fa180-73b1-428f-8f57-fa5d4e66da99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20daa702c71b4d29b0b7d42fb12a3d6c'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:01.045807 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-377fa180-73b1-428f-8f57-fa5d4e66da99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20daa702c71b4d29b0b7d42fb12a3d6c'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:01.055353 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-377fa180-73b1-428f-8f57-fa5d4e66da99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20daa702c71b4d29b0b7d42fb12a3d6c'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:01.056111 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-377fa180-73b1-428f-8f57-fa5d4e66da99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20daa702c71b4d29b0b7d42fb12a3d6c'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:01.056874 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-377fa180-73b1-428f-8f57-fa5d4e66da99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20daa702c71b4d29b0b7d42fb12a3d6c'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:01.075879 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 941/1901] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:00 2022] DELETE /identity/v3/users/20daa702c71b4d29b0b7d42fb12a3d6c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:01.080434 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d19791-f9ab-4d7a-b3be-cdd981bff3d6 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:01.080775 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d19791-f9ab-4d7a-b3be-cdd981bff3d6 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:01.081039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d19791-f9ab-4d7a-b3be-cdd981bff3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:01.099635 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-47d19791-f9ab-4d7a-b3be-cdd981bff3d6 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:01.101118 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-47d19791-f9ab-4d7a-b3be-cdd981bff3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:01.163811 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 961/1902] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:01.173342 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377fa180-73b1-428f-8f57-fa5d4e66da99 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:01.175058 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-377fa180-73b1-428f-8f57-fa5d4e66da99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:01.261301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377fa180-73b1-428f-8f57-fa5d4e66da99 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:01.266853 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1e393-f2ec-4759-b266-0ad6cd80c26f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:01.269026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1e393-f2ec-4759-b266-0ad6cd80c26f service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:01.269632 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1e393-f2ec-4759-b266-0ad6cd80c26f service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:01.269918 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1e393-f2ec-4759-b266-0ad6cd80c26f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:01.281778 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e1e393-f2ec-4759-b266-0ad6cd80c26f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:01.283467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e1e393-f2ec-4759-b266-0ad6cd80c26f service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:01.294515 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 942/1903] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:48:01.734893 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d19791-f9ab-4d7a-b3be-cdd981bff3d6 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:01.737633 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-47d19791-f9ab-4d7a-b3be-cdd981bff3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:01.762028 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d19791-f9ab-4d7a-b3be-cdd981bff3d6 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:01.769753 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc4ddc-3a72-454e-ad85-859eb6acee11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:01.771257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc4ddc-3a72-454e-ad85-859eb6acee11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:01.771540 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc4ddc-3a72-454e-ad85-859eb6acee11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:01.771811 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc4ddc-3a72-454e-ad85-859eb6acee11 admin admin] PATH_INFO: `/v3/projects/28f8c73c343642b099112937878eea43` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:01.775911 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc4ddc-3a72-454e-ad85-859eb6acee11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28f8c73c343642b099112937878eea43)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:01.779375 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc4ddc-3a72-454e-ad85-859eb6acee11 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:01.872208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-94bc4ddc-3a72-454e-ad85-859eb6acee11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28f8c73c343642b099112937878eea43', 'request_id': 'req-94bc4ddc-3a72-454e-ad85-859eb6acee11'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:01.885386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-94bc4ddc-3a72-454e-ad85-859eb6acee11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28f8c73c343642b099112937878eea43', 'request_id': 'req-94bc4ddc-3a72-454e-ad85-859eb6acee11'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:01.895352 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-94bc4ddc-3a72-454e-ad85-859eb6acee11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28f8c73c343642b099112937878eea43', 'request_id': 'req-94bc4ddc-3a72-454e-ad85-859eb6acee11'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:01.913844 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-94bc4ddc-3a72-454e-ad85-859eb6acee11 admin admin] The token cache is being invalidate because project 28f8c73c343642b099112937878eea43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:01.914962 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-94bc4ddc-3a72-454e-ad85-859eb6acee11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:01.916082 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-94bc4ddc-3a72-454e-ad85-859eb6acee11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:01.918893 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 962/1904] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:01 2022] DELETE /identity/v3/projects/28f8c73c343642b099112937878eea43 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:01.922629 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1e393-f2ec-4759-b266-0ad6cd80c26f service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:01.935666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1e393-f2ec-4759-b266-0ad6cd80c26f service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:02.003571 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825c0ee-9691-4aa5-a7e0-0f06f3beafb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:02.004608 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825c0ee-9691-4aa5-a7e0-0f06f3beafb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:02.004909 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825c0ee-9691-4aa5-a7e0-0f06f3beafb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:02.005182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825c0ee-9691-4aa5-a7e0-0f06f3beafb2 admin admin] PATH_INFO: `/v3/users/4f15ef0f7b374345a0f82e3c36bce49c` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:02.006430 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825c0ee-9691-4aa5-a7e0-0f06f3beafb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f15ef0f7b374345a0f82e3c36bce49c)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:02.008614 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825c0ee-9691-4aa5-a7e0-0f06f3beafb2 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:02.080098 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6825c0ee-9691-4aa5-a7e0-0f06f3beafb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f15ef0f7b374345a0f82e3c36bce49c'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:02.087736 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6825c0ee-9691-4aa5-a7e0-0f06f3beafb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f15ef0f7b374345a0f82e3c36bce49c'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:02.099810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6825c0ee-9691-4aa5-a7e0-0f06f3beafb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f15ef0f7b374345a0f82e3c36bce49c'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:02.100509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6825c0ee-9691-4aa5-a7e0-0f06f3beafb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f15ef0f7b374345a0f82e3c36bce49c'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:02.101205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6825c0ee-9691-4aa5-a7e0-0f06f3beafb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f15ef0f7b374345a0f82e3c36bce49c'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:02.119355 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 943/1905] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:01 2022] DELETE /identity/v3/users/4f15ef0f7b374345a0f82e3c36bce49c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:02.401431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc4ddc-3a72-454e-ad85-859eb6acee11 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:02.403274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-94bc4ddc-3a72-454e-ad85-859eb6acee11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:02.425631 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc4ddc-3a72-454e-ad85-859eb6acee11 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:02.502378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388c7f7-599e-492c-b052-43f1ff191853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:02.506721 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388c7f7-599e-492c-b052-43f1ff191853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:02.507236 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388c7f7-599e-492c-b052-43f1ff191853 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:02.507645 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388c7f7-599e-492c-b052-43f1ff191853 admin admin] PATH_INFO: `/v3/projects/5086057dd20d41bb8e2645ada136c422` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:02.510734 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388c7f7-599e-492c-b052-43f1ff191853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5086057dd20d41bb8e2645ada136c422)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:02.514714 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388c7f7-599e-492c-b052-43f1ff191853 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:02.587459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9388c7f7-599e-492c-b052-43f1ff191853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5086057dd20d41bb8e2645ada136c422', 'request_id': 'req-9388c7f7-599e-492c-b052-43f1ff191853'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:02.595316 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9388c7f7-599e-492c-b052-43f1ff191853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5086057dd20d41bb8e2645ada136c422', 'request_id': 'req-9388c7f7-599e-492c-b052-43f1ff191853'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:02.612331 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9388c7f7-599e-492c-b052-43f1ff191853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5086057dd20d41bb8e2645ada136c422', 'request_id': 'req-9388c7f7-599e-492c-b052-43f1ff191853'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:02.627289 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9388c7f7-599e-492c-b052-43f1ff191853 admin admin] The token cache is being invalidate because project 5086057dd20d41bb8e2645ada136c422 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:02.627615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9388c7f7-599e-492c-b052-43f1ff191853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:02.628691 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9388c7f7-599e-492c-b052-43f1ff191853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:02.630918 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 963/1906] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:02 2022] DELETE /identity/v3/projects/5086057dd20d41bb8e2645ada136c422 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:02.635430 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522f6a3-b0df-45b5-abe3-24c5d80d1636 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:02.635712 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522f6a3-b0df-45b5-abe3-24c5d80d1636 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:02.635987 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522f6a3-b0df-45b5-abe3-24c5d80d1636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:02.660875 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-3522f6a3-b0df-45b5-abe3-24c5d80d1636 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:02.662374 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-3522f6a3-b0df-45b5-abe3-24c5d80d1636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:02.769000 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 944/1907] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:02.776903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01421d-7d67-4f45-9f69-2548247a7037 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:02.777266 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01421d-7d67-4f45-9f69-2548247a7037 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:02.777559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01421d-7d67-4f45-9f69-2548247a7037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:02.802109 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-5e01421d-7d67-4f45-9f69-2548247a7037 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:02.803353 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-5e01421d-7d67-4f45-9f69-2548247a7037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:02.818104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 964/1908] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:02.818827 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3522f6a3-b0df-45b5-abe3-24c5d80d1636 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:02.829363 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3522f6a3-b0df-45b5-abe3-24c5d80d1636 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:02.833870 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d4af6-a985-4ca4-8bb6-6f4db08dead4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:02.834785 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d4af6-a985-4ca4-8bb6-6f4db08dead4 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:02.835560 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d4af6-a985-4ca4-8bb6-6f4db08dead4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:02.835560 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d4af6-a985-4ca4-8bb6-6f4db08dead4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:02.836423 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d4af6-a985-4ca4-8bb6-6f4db08dead4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:48:02.837243 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d4af6-a985-4ca4-8bb6-6f4db08dead4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:02.838752 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d4af6-a985-4ca4-8bb6-6f4db08dead4 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:02.853097 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 945/1909] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:48:02.857639 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01421d-7d67-4f45-9f69-2548247a7037 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:02.872000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01421d-7d67-4f45-9f69-2548247a7037 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:02.878197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07296a91-63b2-46c7-a9e7-0a8ecfcac2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:02.879609 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07296a91-63b2-46c7-a9e7-0a8ecfcac2a6 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:02.880070 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07296a91-63b2-46c7-a9e7-0a8ecfcac2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:02.880516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07296a91-63b2-46c7-a9e7-0a8ecfcac2a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:02.881433 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07296a91-63b2-46c7-a9e7-0a8ecfcac2a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:02.883893 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07296a91-63b2-46c7-a9e7-0a8ecfcac2a6 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:02.940403 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 965/1910] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:48:02 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:48:02.945115 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d4af6-a985-4ca4-8bb6-6f4db08dead4 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:02.975244 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d4af6-a985-4ca4-8bb6-6f4db08dead4 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:03.180516 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73f961-2e90-4d58-91c6-f94213355645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:03.183091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73f961-2e90-4d58-91c6-f94213355645 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:03.184075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73f961-2e90-4d58-91c6-f94213355645 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:03.184638 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73f961-2e90-4d58-91c6-f94213355645 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:03.185736 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a73f961-2e90-4d58-91c6-f94213355645 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:03.188149 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a73f961-2e90-4d58-91c6-f94213355645 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:03.253304 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 946/1911] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:48:02 2022] POST /identity/v3/users => generated 287 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:03.253916 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07296a91-63b2-46c7-a9e7-0a8ecfcac2a6 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:03.269735 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07296a91-63b2-46c7-a9e7-0a8ecfcac2a6 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:03.283080 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3218e-4a63-44dd-bc61-81298d05acb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:03.285733 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3218e-4a63-44dd-bc61-81298d05acb0 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:03.286464 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3218e-4a63-44dd-bc61-81298d05acb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:03.289886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3218e-4a63-44dd-bc61-81298d05acb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:03.290834 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3218e-4a63-44dd-bc61-81298d05acb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:03.301412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3218e-4a63-44dd-bc61-81298d05acb0 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:03.326775 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 966/1912] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:03.330858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73f961-2e90-4d58-91c6-f94213355645 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:03.364548 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73f961-2e90-4d58-91c6-f94213355645 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:03.378226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3634f17-6daa-4c5c-a845-98f2cfe1e0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:03.379975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3634f17-6daa-4c5c-a845-98f2cfe1e0c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:03.380278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3634f17-6daa-4c5c-a845-98f2cfe1e0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:03.380547 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3634f17-6daa-4c5c-a845-98f2cfe1e0c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:03.384798 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3634f17-6daa-4c5c-a845-98f2cfe1e0c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:03.386475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3634f17-6daa-4c5c-a845-98f2cfe1e0c9 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:03.412883 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 947/1913] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:03.417245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3218e-4a63-44dd-bc61-81298d05acb0 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:03.435369 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3218e-4a63-44dd-bc61-81298d05acb0 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:03.443942 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f692a3-be39-4bbf-8b19-e333c461fc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:03.445676 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f692a3-be39-4bbf-8b19-e333c461fc9f admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:03.446028 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f692a3-be39-4bbf-8b19-e333c461fc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:03.446308 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f692a3-be39-4bbf-8b19-e333c461fc9f admin admin] PATH_INFO: `/v3/projects/0bf380cd9ee343dd892a2628e8a6e95d/users/1c6864349e01464db81396b388eec082/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:03.479125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f692a3-be39-4bbf-8b19-e333c461fc9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bf380cd9ee343dd892a2628e8a6e95d, user_id=1c6864349e01464db81396b388eec082, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:03.481839 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f692a3-be39-4bbf-8b19-e333c461fc9f admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:03.519376 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 967/1914] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:03 2022] PUT /identity/v3/projects/0bf380cd9ee343dd892a2628e8a6e95d/users/1c6864349e01464db81396b388eec082/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:03.525117 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcc83b-8128-4ccb-ba62-a9ed5b1438eb None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:03.525670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcc83b-8128-4ccb-ba62-a9ed5b1438eb None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:03.526124 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcc83b-8128-4ccb-ba62-a9ed5b1438eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:03.559901 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-4ddcc83b-8128-4ccb-ba62-a9ed5b1438eb None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:03.561270 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-4ddcc83b-8128-4ccb-ba62-a9ed5b1438eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:03.656822 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 948/1915] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:03.669570 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f692a3-be39-4bbf-8b19-e333c461fc9f admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:03.675652 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-06f692a3-be39-4bbf-8b19-e333c461fc9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:03.779508 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f692a3-be39-4bbf-8b19-e333c461fc9f admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:03.789151 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbee796-72e7-465e-b2b6-4fc72f78360a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:03.790047 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbee796-72e7-465e-b2b6-4fc72f78360a service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:03.790467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbee796-72e7-465e-b2b6-4fc72f78360a service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:03.790900 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbee796-72e7-465e-b2b6-4fc72f78360a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:03.804836 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbee796-72e7-465e-b2b6-4fc72f78360a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:03.806529 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbee796-72e7-465e-b2b6-4fc72f78360a service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:03.819246 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 968/1916] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:48:05.954034 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439311a5-0afa-435a-a2f4-d8713bbac280 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:05.955117 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439311a5-0afa-435a-a2f4-d8713bbac280 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:05.955536 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439311a5-0afa-435a-a2f4-d8713bbac280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:06.000971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-439311a5-0afa-435a-a2f4-d8713bbac280 None None] MFA Rules not processed for user `1c6864349e01464db81396b388eec082`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:06.005561 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-439311a5-0afa-435a-a2f4-d8713bbac280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:06.106973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbee796-72e7-465e-b2b6-4fc72f78360a service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:06.109192 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 949/1917] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:06.122630 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbee796-72e7-465e-b2b6-4fc72f78360a service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:06.135991 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf749bf8-2ea0-4ba2-93cf-b6555b6b65fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:06.137479 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf749bf8-2ea0-4ba2-93cf-b6555b6b65fc admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:06.138084 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf749bf8-2ea0-4ba2-93cf-b6555b6b65fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:06.138663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf749bf8-2ea0-4ba2-93cf-b6555b6b65fc admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:06.139648 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf749bf8-2ea0-4ba2-93cf-b6555b6b65fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:06.141960 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf749bf8-2ea0-4ba2-93cf-b6555b6b65fc admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:06.185282 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 969/1918] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:48:06 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:06.187164 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439311a5-0afa-435a-a2f4-d8713bbac280 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:06.202237 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439311a5-0afa-435a-a2f4-d8713bbac280 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:06.214631 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf749bf8-2ea0-4ba2-93cf-b6555b6b65fc admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:06.217124 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-cf749bf8-2ea0-4ba2-93cf-b6555b6b65fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:06.301368 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889be9fa-f829-4d54-bade-f1b49465fd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:06.302318 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889be9fa-f829-4d54-bade-f1b49465fd2b admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:06.302760 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889be9fa-f829-4d54-bade-f1b49465fd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:06.303185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889be9fa-f829-4d54-bade-f1b49465fd2b admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:06.303397 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf749bf8-2ea0-4ba2-93cf-b6555b6b65fc admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:06.304198 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889be9fa-f829-4d54-bade-f1b49465fd2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:06.307471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889be9fa-f829-4d54-bade-f1b49465fd2b admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:06.311256 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99dd38e-3e18-49e4-b2c8-77eada20764c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:06.312774 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99dd38e-3e18-49e4-b2c8-77eada20764c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:06.313287 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99dd38e-3e18-49e4-b2c8-77eada20764c admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:06.313755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99dd38e-3e18-49e4-b2c8-77eada20764c admin admin] PATH_INFO: `/v3/users/e5dc66d5bb814351a94facdc743c2296` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:06.315411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99dd38e-3e18-49e4-b2c8-77eada20764c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5dc66d5bb814351a94facdc743c2296)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:06.317851 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99dd38e-3e18-49e4-b2c8-77eada20764c admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:06.344914 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 950/1919] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:48:06 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:06.349945 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889be9fa-f829-4d54-bade-f1b49465fd2b admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:06.366275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889be9fa-f829-4d54-bade-f1b49465fd2b admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:06.380143 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de2d5d-1003-4f55-8517-2deeb33fa4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:06.383015 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de2d5d-1003-4f55-8517-2deeb33fa4fe admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:06.385370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de2d5d-1003-4f55-8517-2deeb33fa4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:06.386839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de2d5d-1003-4f55-8517-2deeb33fa4fe admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:06.391180 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de2d5d-1003-4f55-8517-2deeb33fa4fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:06.395473 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de2d5d-1003-4f55-8517-2deeb33fa4fe admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:06.423071 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 951/1920] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:48:06.424545 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de2d5d-1003-4f55-8517-2deeb33fa4fe admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:06.426649 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e99dd38e-3e18-49e4-b2c8-77eada20764c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5dc66d5bb814351a94facdc743c2296'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:06.443271 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de2d5d-1003-4f55-8517-2deeb33fa4fe admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:06.450089 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e99dd38e-3e18-49e4-b2c8-77eada20764c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5dc66d5bb814351a94facdc743c2296'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:06.464677 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d006b9b-705e-44e9-80c4-76f800f6f18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:06.464963 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e99dd38e-3e18-49e4-b2c8-77eada20764c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5dc66d5bb814351a94facdc743c2296'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:06.468524 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d006b9b-705e-44e9-80c4-76f800f6f18b admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:06.470150 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d006b9b-705e-44e9-80c4-76f800f6f18b admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:06.470623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d006b9b-705e-44e9-80c4-76f800f6f18b admin admin] PATH_INFO: `/v3/projects/dffda6ea6b0647828f836cbc3d447751/users/a21c88c82ef24286ba81be360dd95a22/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:06.493192 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e99dd38e-3e18-49e4-b2c8-77eada20764c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5dc66d5bb814351a94facdc743c2296'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:06.494066 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e99dd38e-3e18-49e4-b2c8-77eada20764c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5dc66d5bb814351a94facdc743c2296'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:06.508990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d006b9b-705e-44e9-80c4-76f800f6f18b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dffda6ea6b0647828f836cbc3d447751, user_id=a21c88c82ef24286ba81be360dd95a22, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:06.511714 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d006b9b-705e-44e9-80c4-76f800f6f18b admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:06.533666 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 970/1921] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:06 2022] DELETE /identity/v3/users/e5dc66d5bb814351a94facdc743c2296 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:06.538131 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0b1a7-aa5b-437b-be78-42888e4766a5 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:06.538426 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0b1a7-aa5b-437b-be78-42888e4766a5 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:06.538703 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0b1a7-aa5b-437b-be78-42888e4766a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:06.548758 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 952/1922] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:06 2022] PUT /identity/v3/projects/dffda6ea6b0647828f836cbc3d447751/users/a21c88c82ef24286ba81be360dd95a22/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:06.549834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d006b9b-705e-44e9-80c4-76f800f6f18b admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:06.552698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-3d006b9b-705e-44e9-80c4-76f800f6f18b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:06.575260 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-e4e0b1a7-aa5b-437b-be78-42888e4766a5 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:06.576744 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e4e0b1a7-aa5b-437b-be78-42888e4766a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:06.643475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d006b9b-705e-44e9-80c4-76f800f6f18b admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:06.648644 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7538056-f97d-42d9-9b39-c108f2393832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:06.649378 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7538056-f97d-42d9-9b39-c108f2393832 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:06.649670 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7538056-f97d-42d9-9b39-c108f2393832 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:06.649968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7538056-f97d-42d9-9b39-c108f2393832 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:06.650806 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7538056-f97d-42d9-9b39-c108f2393832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:06.651002 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 971/1923] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:06.652470 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7538056-f97d-42d9-9b39-c108f2393832 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:06.660777 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0b1a7-aa5b-437b-be78-42888e4766a5 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:06.663276 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e4e0b1a7-aa5b-437b-be78-42888e4766a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:06.663699 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 953/1924] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:06.669658 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7538056-f97d-42d9-9b39-c108f2393832 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:06.682979 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7538056-f97d-42d9-9b39-c108f2393832 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:06.689354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55539d6-038a-4897-a05d-87d526d72013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:06.690613 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55539d6-038a-4897-a05d-87d526d72013 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:06.690949 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55539d6-038a-4897-a05d-87d526d72013 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:06.691324 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55539d6-038a-4897-a05d-87d526d72013 admin admin] PATH_INFO: `/v3/projects/dffda6ea6b0647828f836cbc3d447751/users/a21c88c82ef24286ba81be360dd95a22/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:06.697543 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55539d6-038a-4897-a05d-87d526d72013 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dffda6ea6b0647828f836cbc3d447751, user_id=a21c88c82ef24286ba81be360dd95a22, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:06.701285 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55539d6-038a-4897-a05d-87d526d72013 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:06.720721 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 954/1925] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:06 2022] PUT /identity/v3/projects/dffda6ea6b0647828f836cbc3d447751/users/a21c88c82ef24286ba81be360dd95a22/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:06.739264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0b1a7-aa5b-437b-be78-42888e4766a5 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:06.826764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91336bea-8f53-45b3-aa3a-b26c9f99f28a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:06.827725 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91336bea-8f53-45b3-aa3a-b26c9f99f28a service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:06.828186 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91336bea-8f53-45b3-aa3a-b26c9f99f28a service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:06.828645 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91336bea-8f53-45b3-aa3a-b26c9f99f28a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:06.922129 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91336bea-8f53-45b3-aa3a-b26c9f99f28a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:06.924016 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91336bea-8f53-45b3-aa3a-b26c9f99f28a service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:06.943659 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 972/1926] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 09:48:07.608577 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a5f1f-ff11-40ab-9989-5873b38637de None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:07.609162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a5f1f-ff11-40ab-9989-5873b38637de None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:07.609162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a5f1f-ff11-40ab-9989-5873b38637de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:07.665626 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-b29a5f1f-ff11-40ab-9989-5873b38637de None None] MFA Rules not processed for user `1c091a71e2ad46ceafe25be874c016a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:07.671290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-b29a5f1f-ff11-40ab-9989-5873b38637de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:07.837683 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 955/1927] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:07.839574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91336bea-8f53-45b3-aa3a-b26c9f99f28a service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:07.841621 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-91336bea-8f53-45b3-aa3a-b26c9f99f28a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:07.856694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91336bea-8f53-45b3-aa3a-b26c9f99f28a service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:07.882847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde8025-59a4-4108-ada8-dd6929779a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:07.883913 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde8025-59a4-4108-ada8-dd6929779a39 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:07.884397 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde8025-59a4-4108-ada8-dd6929779a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:07.884897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde8025-59a4-4108-ada8-dd6929779a39 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:07.885808 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde8025-59a4-4108-ada8-dd6929779a39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:07.890017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde8025-59a4-4108-ada8-dd6929779a39 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:07.956702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a5f1f-ff11-40ab-9989-5873b38637de None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:07.957431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 973/1928] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:48:07 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:07.971190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a5f1f-ff11-40ab-9989-5873b38637de None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:07.998384 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951094a5-7839-4f54-95ee-ca5d9b284c9d None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:07.999863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951094a5-7839-4f54-95ee-ca5d9b284c9d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:08.000378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951094a5-7839-4f54-95ee-ca5d9b284c9d None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:08.002022 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 974/1929] 10.222.0.26 () {54 vars in 921 bytes} [Mon May 9 09:48:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 09:48:08.043516 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc145e9-ee16-42ef-a9af-92b6366ae2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:08.044219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc145e9-ee16-42ef-a9af-92b6366ae2c3 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:08.044497 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc145e9-ee16-42ef-a9af-92b6366ae2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:08.044825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc145e9-ee16-42ef-a9af-92b6366ae2c3 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:08.045592 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc145e9-ee16-42ef-a9af-92b6366ae2c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:08.047161 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dac3d-7c29-4e75-8f00-5946c3dd22e2 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:08.047646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dac3d-7c29-4e75-8f00-5946c3dd22e2 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:08.047843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc145e9-ee16-42ef-a9af-92b6366ae2c3 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:08.048310 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dac3d-7c29-4e75-8f00-5946c3dd22e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:08.077859 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 956/1930] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:48:07 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:48:08.079116 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc145e9-ee16-42ef-a9af-92b6366ae2c3 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:08.084007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-484dac3d-7c29-4e75-8f00-5946c3dd22e2 None None] MFA Rules not processed for user `6362303cb918486c8525225914332ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:08.085574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-484dac3d-7c29-4e75-8f00-5946c3dd22e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:08.090835 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc145e9-ee16-42ef-a9af-92b6366ae2c3 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:08.098276 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a9bb6-c124-4761-941f-41e14a8147c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:08.099320 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a9bb6-c124-4761-941f-41e14a8147c7 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:08.099557 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a9bb6-c124-4761-941f-41e14a8147c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:08.099830 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a9bb6-c124-4761-941f-41e14a8147c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:08.102566 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a9bb6-c124-4761-941f-41e14a8147c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:08.104982 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a9bb6-c124-4761-941f-41e14a8147c7 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:08.122789 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 957/1931] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:08.124297 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a9bb6-c124-4761-941f-41e14a8147c7 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:08.141016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a9bb6-c124-4761-941f-41e14a8147c7 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:08.150295 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52499bb-612b-4d23-9650-91416a0904c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:08.177481 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52499bb-612b-4d23-9650-91416a0904c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:08.178086 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52499bb-612b-4d23-9650-91416a0904c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:08.178272 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52499bb-612b-4d23-9650-91416a0904c0 admin admin] PATH_INFO: `/v3/projects/53e4a501080e4803a7f4e7b2a2cf65ee/users/e536207e603644b69ad040b27970b773/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:08.220381 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 975/1932] 10.222.0.26 () {58 vars in 1018 bytes} [Mon May 9 09:48:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:08.221139 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dac3d-7c29-4e75-8f00-5946c3dd22e2 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:08.222717 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52499bb-612b-4d23-9650-91416a0904c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53e4a501080e4803a7f4e7b2a2cf65ee, user_id=e536207e603644b69ad040b27970b773, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:08.225823 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-484dac3d-7c29-4e75-8f00-5946c3dd22e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:08.227609 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52499bb-612b-4d23-9650-91416a0904c0 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:08.254618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dac3d-7c29-4e75-8f00-5946c3dd22e2 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:08.273382 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 958/1933] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:08 2022] PUT /identity/v3/projects/53e4a501080e4803a7f4e7b2a2cf65ee/users/e536207e603644b69ad040b27970b773/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:08.275780 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52499bb-612b-4d23-9650-91416a0904c0 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:08.385743 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7693856-d78b-4b3b-afc4-1d0af3073849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:08.387130 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7693856-d78b-4b3b-afc4-1d0af3073849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:08.388528 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7693856-d78b-4b3b-afc4-1d0af3073849 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:08.389055 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7693856-d78b-4b3b-afc4-1d0af3073849 admin admin] PATH_INFO: `/v3/projects/16fcb7e90638419cbd62a6f77511ecd0` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:08.391780 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7693856-d78b-4b3b-afc4-1d0af3073849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16fcb7e90638419cbd62a6f77511ecd0)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:08.396870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7693856-d78b-4b3b-afc4-1d0af3073849 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:08.423169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52499bb-612b-4d23-9650-91416a0904c0 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:08.446956 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099137c2-6107-4507-ac30-c969052d4a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:08.449520 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099137c2-6107-4507-ac30-c969052d4a3d admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:08.449913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099137c2-6107-4507-ac30-c969052d4a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:08.450299 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099137c2-6107-4507-ac30-c969052d4a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:08.451437 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099137c2-6107-4507-ac30-c969052d4a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:08.457062 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099137c2-6107-4507-ac30-c969052d4a3d admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:08.483794 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 959/1934] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:08.483794 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099137c2-6107-4507-ac30-c969052d4a3d admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:08.489093 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-099137c2-6107-4507-ac30-c969052d4a3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:08.524341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-b7693856-d78b-4b3b-afc4-1d0af3073849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16fcb7e90638419cbd62a6f77511ecd0', 'request_id': 'req-b7693856-d78b-4b3b-afc4-1d0af3073849'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:08.544982 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-b7693856-d78b-4b3b-afc4-1d0af3073849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16fcb7e90638419cbd62a6f77511ecd0', 'request_id': 'req-b7693856-d78b-4b3b-afc4-1d0af3073849'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:08.559603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-b7693856-d78b-4b3b-afc4-1d0af3073849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16fcb7e90638419cbd62a6f77511ecd0', 'request_id': 'req-b7693856-d78b-4b3b-afc4-1d0af3073849'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:08.584836 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-b7693856-d78b-4b3b-afc4-1d0af3073849 admin admin] The token cache is being invalidate because project 16fcb7e90638419cbd62a6f77511ecd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:08.585425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-b7693856-d78b-4b3b-afc4-1d0af3073849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:08.587153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-b7693856-d78b-4b3b-afc4-1d0af3073849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:08.591342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 976/1935] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:08 2022] DELETE /identity/v3/projects/16fcb7e90638419cbd62a6f77511ecd0 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:08.592284 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7693856-d78b-4b3b-afc4-1d0af3073849 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:08.601807 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b7693856-d78b-4b3b-afc4-1d0af3073849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:08.636192 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099137c2-6107-4507-ac30-c969052d4a3d admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:08.724829 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7693856-d78b-4b3b-afc4-1d0af3073849 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:08.729872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbd179-6d6c-4bf7-8f85-17076ced6a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:08.730852 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbd179-6d6c-4bf7-8f85-17076ced6a21 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:08.731197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbd179-6d6c-4bf7-8f85-17076ced6a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:08.731528 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbd179-6d6c-4bf7-8f85-17076ced6a21 admin admin] PATH_INFO: `/v3/projects/53e4a501080e4803a7f4e7b2a2cf65ee/users/e536207e603644b69ad040b27970b773/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:08.733764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbd179-6d6c-4bf7-8f85-17076ced6a21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53e4a501080e4803a7f4e7b2a2cf65ee, user_id=e536207e603644b69ad040b27970b773, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:08.736951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbd179-6d6c-4bf7-8f85-17076ced6a21 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:08.747101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98766c2-5104-4626-ad71-b9eb95be946e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:08.748176 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98766c2-5104-4626-ad71-b9eb95be946e service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:08.748637 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98766c2-5104-4626-ad71-b9eb95be946e service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:08.749607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98766c2-5104-4626-ad71-b9eb95be946e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:08.751881 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a98766c2-5104-4626-ad71-b9eb95be946e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:08.756532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 977/1936] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:08 2022] PUT /identity/v3/projects/53e4a501080e4803a7f4e7b2a2cf65ee/users/e536207e603644b69ad040b27970b773/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:08.757542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbd179-6d6c-4bf7-8f85-17076ced6a21 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:08.762643 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-d3cbd179-6d6c-4bf7-8f85-17076ced6a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:08.770166 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98766c2-5104-4626-ad71-b9eb95be946e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:08.773354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98766c2-5104-4626-ad71-b9eb95be946e service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:08.788118 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 960/1937] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:48:08.837860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbd179-6d6c-4bf7-8f85-17076ced6a21 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:08.842711 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8a024-fd49-4140-a025-16b9cdafdd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:08.843587 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8a024-fd49-4140-a025-16b9cdafdd84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:08.843869 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8a024-fd49-4140-a025-16b9cdafdd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:08.844142 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8a024-fd49-4140-a025-16b9cdafdd84 admin admin] PATH_INFO: `/v3/users/b55f70c0a7964afea131d9ec1d984fc2` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:08.845372 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8a024-fd49-4140-a025-16b9cdafdd84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b55f70c0a7964afea131d9ec1d984fc2)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:08.847704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8a024-fd49-4140-a025-16b9cdafdd84 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:08.905305 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-d5b8a024-fd49-4140-a025-16b9cdafdd84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b55f70c0a7964afea131d9ec1d984fc2'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:08.913013 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-d5b8a024-fd49-4140-a025-16b9cdafdd84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b55f70c0a7964afea131d9ec1d984fc2'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:08.913899 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-d5b8a024-fd49-4140-a025-16b9cdafdd84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b55f70c0a7964afea131d9ec1d984fc2'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:08.923208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-d5b8a024-fd49-4140-a025-16b9cdafdd84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b55f70c0a7964afea131d9ec1d984fc2'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:08.924152 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-d5b8a024-fd49-4140-a025-16b9cdafdd84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b55f70c0a7964afea131d9ec1d984fc2'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:08.946128 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 978/1938] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:08 2022] DELETE /identity/v3/users/b55f70c0a7964afea131d9ec1d984fc2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:09.290393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a049a0-b8b7-414a-9c37-ba423a419cc8 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:09.290685 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a049a0-b8b7-414a-9c37-ba423a419cc8 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:09.290965 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a049a0-b8b7-414a-9c37-ba423a419cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:09.357807 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-e9a049a0-b8b7-414a-9c37-ba423a419cc8 None None] MFA Rules not processed for user `a21c88c82ef24286ba81be360dd95a22`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:09.359432 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e9a049a0-b8b7-414a-9c37-ba423a419cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:09.463887 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 961/1939] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:09 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:09.471653 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8a024-fd49-4140-a025-16b9cdafdd84 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:09.476735 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-d5b8a024-fd49-4140-a025-16b9cdafdd84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:09.500271 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a049a0-b8b7-414a-9c37-ba423a419cc8 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:09.504628 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e9a049a0-b8b7-414a-9c37-ba423a419cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:09.610010 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8a024-fd49-4140-a025-16b9cdafdd84 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:09.627206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18eaf6c-9091-4e0e-9803-3b3a252ea720 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:09.627823 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18eaf6c-9091-4e0e-9803-3b3a252ea720 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:09.628473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18eaf6c-9091-4e0e-9803-3b3a252ea720 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:09.628473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18eaf6c-9091-4e0e-9803-3b3a252ea720 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:09.638342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-f18eaf6c-9091-4e0e-9803-3b3a252ea720 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:09.652606 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a049a0-b8b7-414a-9c37-ba423a419cc8 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:09.659247 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7bbc1a-4cb6-4ebf-bdb2-28246771a85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:09.660872 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7bbc1a-4cb6-4ebf-bdb2-28246771a85b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:09.661361 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7bbc1a-4cb6-4ebf-bdb2-28246771a85b admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:09.661832 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7bbc1a-4cb6-4ebf-bdb2-28246771a85b admin admin] PATH_INFO: `/v3/projects/153df24b6b9c4bc7937e0e94bdc8fdf0` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:09.663271 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7bbc1a-4cb6-4ebf-bdb2-28246771a85b admin admin] RBAC: Authorizing `identity:delete_project(project_id=153df24b6b9c4bc7937e0e94bdc8fdf0)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:09.665456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7bbc1a-4cb6-4ebf-bdb2-28246771a85b admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:09.749522 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18eaf6c-9091-4e0e-9803-3b3a252ea720 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:09.750985 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18eaf6c-9091-4e0e-9803-3b3a252ea720 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:09.756764 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-9f7bbc1a-4cb6-4ebf-bdb2-28246771a85b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '153df24b6b9c4bc7937e0e94bdc8fdf0', 'request_id': 'req-9f7bbc1a-4cb6-4ebf-bdb2-28246771a85b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:09.764227 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 979/1940] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:48:09.765245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18eaf6c-9091-4e0e-9803-3b3a252ea720 service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:09.767014 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-f18eaf6c-9091-4e0e-9803-3b3a252ea720 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:09.786184 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-9f7bbc1a-4cb6-4ebf-bdb2-28246771a85b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '153df24b6b9c4bc7937e0e94bdc8fdf0', 'request_id': 'req-9f7bbc1a-4cb6-4ebf-bdb2-28246771a85b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:09.795631 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-9f7bbc1a-4cb6-4ebf-bdb2-28246771a85b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '153df24b6b9c4bc7937e0e94bdc8fdf0', 'request_id': 'req-9f7bbc1a-4cb6-4ebf-bdb2-28246771a85b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:09.800247 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18eaf6c-9091-4e0e-9803-3b3a252ea720 service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:09.822479 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-9f7bbc1a-4cb6-4ebf-bdb2-28246771a85b admin admin] The token cache is being invalidate because project 153df24b6b9c4bc7937e0e94bdc8fdf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:09.823156 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-9f7bbc1a-4cb6-4ebf-bdb2-28246771a85b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:09.823869 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-9f7bbc1a-4cb6-4ebf-bdb2-28246771a85b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:09.825973 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 962/1941] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:09 2022] DELETE /identity/v3/projects/153df24b6b9c4bc7937e0e94bdc8fdf0 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:09.831624 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d6b03-3065-4afc-9725-8f3fd0ffd0a9 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:09.831922 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d6b03-3065-4afc-9725-8f3fd0ffd0a9 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:09.832188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d6b03-3065-4afc-9725-8f3fd0ffd0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:09.858644 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-e14d6b03-3065-4afc-9725-8f3fd0ffd0a9 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:09.861170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e14d6b03-3065-4afc-9725-8f3fd0ffd0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:09.878765 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2580c-f875-4581-b9d1-aa82d0d0cfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:09.879660 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2580c-f875-4581-b9d1-aa82d0d0cfcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:09.879937 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2580c-f875-4581-b9d1-aa82d0d0cfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:09.880209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2580c-f875-4581-b9d1-aa82d0d0cfcd admin admin] PATH_INFO: `/v3/users/d1030b74c60c4a38bb563f69c3474e0e` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:09.881603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2580c-f875-4581-b9d1-aa82d0d0cfcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1030b74c60c4a38bb563f69c3474e0e)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:09.884136 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2580c-f875-4581-b9d1-aa82d0d0cfcd admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:09.968400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-61b2580c-f875-4581-b9d1-aa82d0d0cfcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1030b74c60c4a38bb563f69c3474e0e'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:09.980904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-61b2580c-f875-4581-b9d1-aa82d0d0cfcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1030b74c60c4a38bb563f69c3474e0e'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:09.981757 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-61b2580c-f875-4581-b9d1-aa82d0d0cfcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1030b74c60c4a38bb563f69c3474e0e'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:09.998030 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 963/1942] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:09.998339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-61b2580c-f875-4581-b9d1-aa82d0d0cfcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1030b74c60c4a38bb563f69c3474e0e'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:09.998339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-61b2580c-f875-4581-b9d1-aa82d0d0cfcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1030b74c60c4a38bb563f69c3474e0e'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:10.000591 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88db62e-f717-44f6-8a26-fad59cf9d727 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:10.002206 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88db62e-f717-44f6-8a26-fad59cf9d727 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:10.002489 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88db62e-f717-44f6-8a26-fad59cf9d727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:10.030486 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 980/1943] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:09 2022] DELETE /identity/v3/users/d1030b74c60c4a38bb563f69c3474e0e => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:10.031938 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-f88db62e-f717-44f6-8a26-fad59cf9d727 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:10.033383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f88db62e-f717-44f6-8a26-fad59cf9d727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:10.035052 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ee0c6-8deb-4e73-8621-d4c22c63795c None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:10.035339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ee0c6-8deb-4e73-8621-d4c22c63795c None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:10.035605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ee0c6-8deb-4e73-8621-d4c22c63795c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:10.093454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-3c5ee0c6-8deb-4e73-8621-d4c22c63795c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:10.104967 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-3c5ee0c6-8deb-4e73-8621-d4c22c63795c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:10.169667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 964/1944] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:10.174336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88db62e-f717-44f6-8a26-fad59cf9d727 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:10.179706 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 981/1945] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:10.192160 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ee0c6-8deb-4e73-8621-d4c22c63795c None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:10.195360 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-3c5ee0c6-8deb-4e73-8621-d4c22c63795c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:10.204565 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88db62e-f717-44f6-8a26-fad59cf9d727 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:10.222429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192eb5d-2253-49d9-94f1-4171ba63fea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:10.225183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192eb5d-2253-49d9-94f1-4171ba63fea6 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:10.225469 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192eb5d-2253-49d9-94f1-4171ba63fea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:10.226469 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192eb5d-2253-49d9-94f1-4171ba63fea6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:10.228260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192eb5d-2253-49d9-94f1-4171ba63fea6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:48:10.229722 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192eb5d-2253-49d9-94f1-4171ba63fea6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:10.235739 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192eb5d-2253-49d9-94f1-4171ba63fea6 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:10.277818 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 965/1946] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:48:10.280066 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192eb5d-2253-49d9-94f1-4171ba63fea6 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:10.284527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ee0c6-8deb-4e73-8621-d4c22c63795c None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:10.303574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831bb2f-1201-4fb0-beb5-fc09afefdac3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:10.305403 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831bb2f-1201-4fb0-beb5-fc09afefdac3 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:10.306421 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831bb2f-1201-4fb0-beb5-fc09afefdac3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:10.306711 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831bb2f-1201-4fb0-beb5-fc09afefdac3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:10.321073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192eb5d-2253-49d9-94f1-4171ba63fea6 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:10.336748 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008289aa-2e53-4c99-9360-590cbd72fb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:10.337457 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008289aa-2e53-4c99-9360-590cbd72fb77 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:10.337850 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008289aa-2e53-4c99-9360-590cbd72fb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:10.338161 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008289aa-2e53-4c99-9360-590cbd72fb77 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:10.339084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008289aa-2e53-4c99-9360-590cbd72fb77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:10.341092 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008289aa-2e53-4c99-9360-590cbd72fb77 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:10.342876 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831bb2f-1201-4fb0-beb5-fc09afefdac3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:10.344542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831bb2f-1201-4fb0-beb5-fc09afefdac3 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:10.379802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 982/1947] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 09:48:10.407159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 966/1948] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:48:10 2022] POST /identity/v3/projects => generated 389 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:10.411545 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831bb2f-1201-4fb0-beb5-fc09afefdac3 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:10.426049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831bb2f-1201-4fb0-beb5-fc09afefdac3 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:10.545116 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7706966-58cd-43ff-96e6-4e8d62846c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:10.546085 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7706966-58cd-43ff-96e6-4e8d62846c2b admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:10.548953 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7706966-58cd-43ff-96e6-4e8d62846c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:10.549486 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7706966-58cd-43ff-96e6-4e8d62846c2b admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:10.551229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7706966-58cd-43ff-96e6-4e8d62846c2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:10.561169 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7706966-58cd-43ff-96e6-4e8d62846c2b admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:10.593135 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 983/1949] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:48:10 2022] POST /identity/v3/users => generated 295 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:10.594540 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008289aa-2e53-4c99-9360-590cbd72fb77 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:10.606912 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008289aa-2e53-4c99-9360-590cbd72fb77 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:10.613601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0874f2c-b7ed-4460-a261-0070c77e2db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:10.614588 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0874f2c-b7ed-4460-a261-0070c77e2db8 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:10.615154 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0874f2c-b7ed-4460-a261-0070c77e2db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:10.615573 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0874f2c-b7ed-4460-a261-0070c77e2db8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:10.616602 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0874f2c-b7ed-4460-a261-0070c77e2db8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:10.618594 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0874f2c-b7ed-4460-a261-0070c77e2db8 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:10.642666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 967/1950] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:10.644767 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7706966-58cd-43ff-96e6-4e8d62846c2b admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:10.659403 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7706966-58cd-43ff-96e6-4e8d62846c2b admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:10.677945 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb34b4-d5be-4da5-bf4a-d258ad66784c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:10.678162 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb34b4-d5be-4da5-bf4a-d258ad66784c admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:10.678359 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb34b4-d5be-4da5-bf4a-d258ad66784c admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:10.678610 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb34b4-d5be-4da5-bf4a-d258ad66784c admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:10.680000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb34b4-d5be-4da5-bf4a-d258ad66784c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:10.681685 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb34b4-d5be-4da5-bf4a-d258ad66784c admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:10.712706 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0874f2c-b7ed-4460-a261-0070c77e2db8 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:10.720605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 984/1951] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:10.724318 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0874f2c-b7ed-4460-a261-0070c77e2db8 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:10.741040 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f84f37-ded9-4208-88c9-f05cfe973fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:10.742362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f84f37-ded9-4208-88c9-f05cfe973fe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:10.742362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f84f37-ded9-4208-88c9-f05cfe973fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:10.742568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f84f37-ded9-4208-88c9-f05cfe973fe5 admin admin] PATH_INFO: `/v3/projects/9817d732b1e44b828041ac05f0d41e18/users/7a1e92235f7a4ac18f848082aa8f1dbf/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:10.764302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f84f37-ded9-4208-88c9-f05cfe973fe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9817d732b1e44b828041ac05f0d41e18, user_id=7a1e92235f7a4ac18f848082aa8f1dbf, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:10.767111 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f84f37-ded9-4208-88c9-f05cfe973fe5 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:10.809856 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49e588-07fb-42f8-bcd7-f3400e3b3095 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:10.810140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49e588-07fb-42f8-bcd7-f3400e3b3095 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:10.810464 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49e588-07fb-42f8-bcd7-f3400e3b3095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:10.812538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 968/1952] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:10 2022] PUT /identity/v3/projects/9817d732b1e44b828041ac05f0d41e18/users/7a1e92235f7a4ac18f848082aa8f1dbf/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:10.878921 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-3e49e588-07fb-42f8-bcd7-f3400e3b3095 None None] MFA Rules not processed for user `7a1e92235f7a4ac18f848082aa8f1dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:10.880245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-3e49e588-07fb-42f8-bcd7-f3400e3b3095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:10.975914 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 985/1953] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:10.980323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f84f37-ded9-4208-88c9-f05cfe973fe5 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:11.037956 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49e588-07fb-42f8-bcd7-f3400e3b3095 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:11.038786 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-3e49e588-07fb-42f8-bcd7-f3400e3b3095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:11.144854 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f84f37-ded9-4208-88c9-f05cfe973fe5 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:11.150922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49e588-07fb-42f8-bcd7-f3400e3b3095 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:11.155901 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e2d6a-2582-49cc-977a-b12d78198a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:11.156822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e2d6a-2582-49cc-977a-b12d78198a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:11.157104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e2d6a-2582-49cc-977a-b12d78198a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:11.157372 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e2d6a-2582-49cc-977a-b12d78198a9e admin admin] PATH_INFO: `/v3/projects/b53f4e48011e4f019fa99c1ac0dd307e` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:11.158693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e2d6a-2582-49cc-977a-b12d78198a9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b53f4e48011e4f019fa99c1ac0dd307e)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:11.159839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc665b-e94b-4ea5-b7dc-9b0cb3a4ff79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:11.160708 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc665b-e94b-4ea5-b7dc-9b0cb3a4ff79 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:11.160953 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e2d6a-2582-49cc-977a-b12d78198a9e admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:11.161067 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc665b-e94b-4ea5-b7dc-9b0cb3a4ff79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:11.161257 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc665b-e94b-4ea5-b7dc-9b0cb3a4ff79 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:11.161962 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc665b-e94b-4ea5-b7dc-9b0cb3a4ff79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:11.163863 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc665b-e94b-4ea5-b7dc-9b0cb3a4ff79 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:11.208105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 969/1954] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:48:10 2022] POST /identity/v3/projects => generated 389 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:11.212254 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc665b-e94b-4ea5-b7dc-9b0cb3a4ff79 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:11.225188 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0c9e2d6a-2582-49cc-977a-b12d78198a9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b53f4e48011e4f019fa99c1ac0dd307e', 'request_id': 'req-0c9e2d6a-2582-49cc-977a-b12d78198a9e'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:11.237957 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0c9e2d6a-2582-49cc-977a-b12d78198a9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b53f4e48011e4f019fa99c1ac0dd307e', 'request_id': 'req-0c9e2d6a-2582-49cc-977a-b12d78198a9e'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:11.244535 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0c9e2d6a-2582-49cc-977a-b12d78198a9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b53f4e48011e4f019fa99c1ac0dd307e', 'request_id': 'req-0c9e2d6a-2582-49cc-977a-b12d78198a9e'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:11.271214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0c9e2d6a-2582-49cc-977a-b12d78198a9e admin admin] The token cache is being invalidate because project b53f4e48011e4f019fa99c1ac0dd307e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:11.272444 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0c9e2d6a-2582-49cc-977a-b12d78198a9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:11.273777 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0c9e2d6a-2582-49cc-977a-b12d78198a9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:11.277419 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 986/1955] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:11 2022] DELETE /identity/v3/projects/b53f4e48011e4f019fa99c1ac0dd307e => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:11.281393 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e2d6a-2582-49cc-977a-b12d78198a9e admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:11.284013 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-0c9e2d6a-2582-49cc-977a-b12d78198a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:11.312889 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc665b-e94b-4ea5-b7dc-9b0cb3a4ff79 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:11.319799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555160d0-10f4-4a97-91eb-89172c744a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:11.320565 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555160d0-10f4-4a97-91eb-89172c744a1e admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:11.321322 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555160d0-10f4-4a97-91eb-89172c744a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:11.322344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555160d0-10f4-4a97-91eb-89172c744a1e admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:11.324016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555160d0-10f4-4a97-91eb-89172c744a1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:11.326308 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555160d0-10f4-4a97-91eb-89172c744a1e admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:11.359912 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 970/1956] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:48:11 2022] POST /identity/v3/users => generated 301 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:11.361699 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555160d0-10f4-4a97-91eb-89172c744a1e admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:11.364535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-555160d0-10f4-4a97-91eb-89172c744a1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:11.378765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555160d0-10f4-4a97-91eb-89172c744a1e admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:11.379940 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e2d6a-2582-49cc-977a-b12d78198a9e admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:11.386240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba45a2-374e-43c5-a669-4e0b5e869f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:11.388117 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2058e0-924a-4c89-bdac-39bfbcde0588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:11.390649 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba45a2-374e-43c5-a669-4e0b5e869f13 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:11.391443 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba45a2-374e-43c5-a669-4e0b5e869f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:11.391443 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba45a2-374e-43c5-a669-4e0b5e869f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:11.392455 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba45a2-374e-43c5-a669-4e0b5e869f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:11.395373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba45a2-374e-43c5-a669-4e0b5e869f13 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:11.407642 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2058e0-924a-4c89-bdac-39bfbcde0588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:11.408216 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2058e0-924a-4c89-bdac-39bfbcde0588 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:11.409338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2058e0-924a-4c89-bdac-39bfbcde0588 admin admin] PATH_INFO: `/v3/users/589a9be4be034a00b1b8d75923b8088e` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:11.415974 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 971/1957] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:48:11.419504 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba45a2-374e-43c5-a669-4e0b5e869f13 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:11.421038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2058e0-924a-4c89-bdac-39bfbcde0588 admin admin] RBAC: Authorizing `identity:delete_user(user_id=589a9be4be034a00b1b8d75923b8088e)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:11.427429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2058e0-924a-4c89-bdac-39bfbcde0588 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:11.444032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba45a2-374e-43c5-a669-4e0b5e869f13 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:11.454379 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af0dd8-e260-47c8-b9ac-f6ebd51c5962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:11.455307 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af0dd8-e260-47c8-b9ac-f6ebd51c5962 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:11.457282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af0dd8-e260-47c8-b9ac-f6ebd51c5962 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:11.457541 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af0dd8-e260-47c8-b9ac-f6ebd51c5962 admin admin] PATH_INFO: `/v3/projects/20c9d825bb0a48d09e3d278ef395f1a6/users/a0d6261c01fe461680d63fcb461e51bf/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:11.491756 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af0dd8-e260-47c8-b9ac-f6ebd51c5962 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c9d825bb0a48d09e3d278ef395f1a6, user_id=a0d6261c01fe461680d63fcb461e51bf, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:11.495694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af0dd8-e260-47c8-b9ac-f6ebd51c5962 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:11.523385 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 972/1958] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:11 2022] PUT /identity/v3/projects/20c9d825bb0a48d09e3d278ef395f1a6/users/a0d6261c01fe461680d63fcb461e51bf/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:48:11.524783 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af0dd8-e260-47c8-b9ac-f6ebd51c5962 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:11.526472 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-c6af0dd8-e260-47c8-b9ac-f6ebd51c5962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:11.535674 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0d2058e0-924a-4c89-bdac-39bfbcde0588 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '589a9be4be034a00b1b8d75923b8088e'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:11.550536 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0d2058e0-924a-4c89-bdac-39bfbcde0588 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '589a9be4be034a00b1b8d75923b8088e'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:11.551432 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0d2058e0-924a-4c89-bdac-39bfbcde0588 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '589a9be4be034a00b1b8d75923b8088e'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:11.569190 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0d2058e0-924a-4c89-bdac-39bfbcde0588 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '589a9be4be034a00b1b8d75923b8088e'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:11.570823 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-0d2058e0-924a-4c89-bdac-39bfbcde0588 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '589a9be4be034a00b1b8d75923b8088e'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:11.598169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af0dd8-e260-47c8-b9ac-f6ebd51c5962 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:11.600200 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 987/1959] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:11 2022] DELETE /identity/v3/users/589a9be4be034a00b1b8d75923b8088e => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:48:11.601184 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2058e0-924a-4c89-bdac-39bfbcde0588 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:11.603392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-0d2058e0-924a-4c89-bdac-39bfbcde0588 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:11.662527 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13dac6-b9de-4c64-81be-ae1e54b08d50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:11.663542 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13dac6-b9de-4c64-81be-ae1e54b08d50 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:11.664212 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13dac6-b9de-4c64-81be-ae1e54b08d50 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:11.664717 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13dac6-b9de-4c64-81be-ae1e54b08d50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:11.668390 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-fa13dac6-b9de-4c64-81be-ae1e54b08d50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:11.712297 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2058e0-924a-4c89-bdac-39bfbcde0588 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:11.718212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01577cc6-9be4-4a92-bda0-595cfb36f257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:11.718941 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01577cc6-9be4-4a92-bda0-595cfb36f257 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:11.719155 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01577cc6-9be4-4a92-bda0-595cfb36f257 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:11.719425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01577cc6-9be4-4a92-bda0-595cfb36f257 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:11.720150 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01577cc6-9be4-4a92-bda0-595cfb36f257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:11.722367 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01577cc6-9be4-4a92-bda0-595cfb36f257 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:11.732226 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 988/1960] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:11.736642 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01577cc6-9be4-4a92-bda0-595cfb36f257 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:11.747623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01577cc6-9be4-4a92-bda0-595cfb36f257 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:11.753289 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158257e7-e12d-4bb9-bf2a-509ab6dd352c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:11.754537 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158257e7-e12d-4bb9-bf2a-509ab6dd352c admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:11.754845 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158257e7-e12d-4bb9-bf2a-509ab6dd352c admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:11.755143 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158257e7-e12d-4bb9-bf2a-509ab6dd352c admin admin] PATH_INFO: `/v3/projects/20c9d825bb0a48d09e3d278ef395f1a6/users/a0d6261c01fe461680d63fcb461e51bf/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:11.757480 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158257e7-e12d-4bb9-bf2a-509ab6dd352c admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c9d825bb0a48d09e3d278ef395f1a6, user_id=a0d6261c01fe461680d63fcb461e51bf, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:11.766030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158257e7-e12d-4bb9-bf2a-509ab6dd352c admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:11.768249 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa13dac6-b9de-4c64-81be-ae1e54b08d50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:11.771405 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa13dac6-b9de-4c64-81be-ae1e54b08d50 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:11.787313 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 973/1961] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:11 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:48:11.806375 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 989/1962] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:11 2022] PUT /identity/v3/projects/20c9d825bb0a48d09e3d278ef395f1a6/users/a0d6261c01fe461680d63fcb461e51bf/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:11.811789 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30187e2-4e11-48b3-b6fc-bf87e15c1bc9 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:11.812358 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30187e2-4e11-48b3-b6fc-bf87e15c1bc9 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:11.812735 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30187e2-4e11-48b3-b6fc-bf87e15c1bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:11.858981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-a30187e2-4e11-48b3-b6fc-bf87e15c1bc9 None None] MFA Rules not processed for user `a0d6261c01fe461680d63fcb461e51bf`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:11.862343 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a30187e2-4e11-48b3-b6fc-bf87e15c1bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:11.972056 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 974/1963] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:11 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:11.983444 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158257e7-e12d-4bb9-bf2a-509ab6dd352c admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:11.986296 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-158257e7-e12d-4bb9-bf2a-509ab6dd352c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:12.069825 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158257e7-e12d-4bb9-bf2a-509ab6dd352c admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:12.074837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c873d0-3982-4607-951a-abbb90513ed2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:12.076241 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c873d0-3982-4607-951a-abbb90513ed2 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:12.076938 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c873d0-3982-4607-951a-abbb90513ed2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:12.077439 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c873d0-3982-4607-951a-abbb90513ed2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:12.096016 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c873d0-3982-4607-951a-abbb90513ed2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:12.098709 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c873d0-3982-4607-951a-abbb90513ed2 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:12.113274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 990/1964] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:11 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:48:12.283616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30187e2-4e11-48b3-b6fc-bf87e15c1bc9 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:12.378547 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30187e2-4e11-48b3-b6fc-bf87e15c1bc9 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:12.386616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0fdc3-c32d-42db-875f-fccc8f78b6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:12.387642 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0fdc3-c32d-42db-875f-fccc8f78b6fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:12.388061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0fdc3-c32d-42db-875f-fccc8f78b6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:12.388612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0fdc3-c32d-42db-875f-fccc8f78b6fd admin admin] PATH_INFO: `/v3/users/7a1e92235f7a4ac18f848082aa8f1dbf` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:12.390629 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0fdc3-c32d-42db-875f-fccc8f78b6fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a1e92235f7a4ac18f848082aa8f1dbf)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:12.392802 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0fdc3-c32d-42db-875f-fccc8f78b6fd admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:12.453019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c873d0-3982-4607-951a-abbb90513ed2 service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:12.455100 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-f6c873d0-3982-4607-951a-abbb90513ed2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:12.476642 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c873d0-3982-4607-951a-abbb90513ed2 service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:12.481249 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-a1a0fdc3-c32d-42db-875f-fccc8f78b6fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a1e92235f7a4ac18f848082aa8f1dbf'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:12.486586 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e5968-9ce5-4bc0-bd11-3ffee582fe7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:12.488106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e5968-9ce5-4bc0-bd11-3ffee582fe7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:12.488594 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e5968-9ce5-4bc0-bd11-3ffee582fe7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:12.489112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e5968-9ce5-4bc0-bd11-3ffee582fe7c admin admin] PATH_INFO: `/v3/projects/5a1114401b21495da8f9d02cd31146c8` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:12.489229 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-a1a0fdc3-c32d-42db-875f-fccc8f78b6fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a1e92235f7a4ac18f848082aa8f1dbf'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:12.491180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e5968-9ce5-4bc0-bd11-3ffee582fe7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a1114401b21495da8f9d02cd31146c8)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:12.493380 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e5968-9ce5-4bc0-bd11-3ffee582fe7c admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:12.499911 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-a1a0fdc3-c32d-42db-875f-fccc8f78b6fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a1e92235f7a4ac18f848082aa8f1dbf'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:12.500878 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-a1a0fdc3-c32d-42db-875f-fccc8f78b6fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a1e92235f7a4ac18f848082aa8f1dbf'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:12.501631 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-a1a0fdc3-c32d-42db-875f-fccc8f78b6fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a1e92235f7a4ac18f848082aa8f1dbf'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:12.522278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 975/1965] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:12 2022] DELETE /identity/v3/users/7a1e92235f7a4ac18f848082aa8f1dbf => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:12.534328 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b9ea6-c785-468e-9a7f-da9231e6af97 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:12.534659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b9ea6-c785-468e-9a7f-da9231e6af97 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:12.535028 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b9ea6-c785-468e-9a7f-da9231e6af97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:12.588181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-5f3e5968-9ce5-4bc0-bd11-3ffee582fe7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a1114401b21495da8f9d02cd31146c8', 'request_id': 'req-5f3e5968-9ce5-4bc0-bd11-3ffee582fe7c'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:12.599383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-5f3e5968-9ce5-4bc0-bd11-3ffee582fe7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a1114401b21495da8f9d02cd31146c8', 'request_id': 'req-5f3e5968-9ce5-4bc0-bd11-3ffee582fe7c'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:12.605989 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-068b9ea6-c785-468e-9a7f-da9231e6af97 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:12.606163 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-5f3e5968-9ce5-4bc0-bd11-3ffee582fe7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a1114401b21495da8f9d02cd31146c8', 'request_id': 'req-5f3e5968-9ce5-4bc0-bd11-3ffee582fe7c'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:12.607311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-068b9ea6-c785-468e-9a7f-da9231e6af97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:12.629311 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-5f3e5968-9ce5-4bc0-bd11-3ffee582fe7c admin admin] The token cache is being invalidate because project 5a1114401b21495da8f9d02cd31146c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:12.629650 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-5f3e5968-9ce5-4bc0-bd11-3ffee582fe7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:12.630476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-5f3e5968-9ce5-4bc0-bd11-3ffee582fe7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:12.633849 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 991/1966] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:12 2022] DELETE /identity/v3/projects/5a1114401b21495da8f9d02cd31146c8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:12.639454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac646d87-2bf3-4057-9b0f-d3379ea3c7e3 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:12.640031 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac646d87-2bf3-4057-9b0f-d3379ea3c7e3 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:12.640309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac646d87-2bf3-4057-9b0f-d3379ea3c7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:12.688153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-ac646d87-2bf3-4057-9b0f-d3379ea3c7e3 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:12.689998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-ac646d87-2bf3-4057-9b0f-d3379ea3c7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:12.729050 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 976/1967] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:12.736379 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b9ea6-c785-468e-9a7f-da9231e6af97 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:12.738993 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-068b9ea6-c785-468e-9a7f-da9231e6af97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:12.788041 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 992/1968] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:48:12.797200 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2754080-1ba1-46b4-962b-f9158b972ec2 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:12.797481 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2754080-1ba1-46b4-962b-f9158b972ec2 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:12.797767 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2754080-1ba1-46b4-962b-f9158b972ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:12.822332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-e2754080-1ba1-46b4-962b-f9158b972ec2 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:12.823575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e2754080-1ba1-46b4-962b-f9158b972ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:12.840689 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 993/1969] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:12.842211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2754080-1ba1-46b4-962b-f9158b972ec2 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:12.851817 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2754080-1ba1-46b4-962b-f9158b972ec2 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:12.858037 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adfc5c-024d-44b1-a451-1905b702a822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:12.858811 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adfc5c-024d-44b1-a451-1905b702a822 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:12.859093 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adfc5c-024d-44b1-a451-1905b702a822 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:12.859369 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adfc5c-024d-44b1-a451-1905b702a822 admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:12.860165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0adfc5c-024d-44b1-a451-1905b702a822 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:48:12.860871 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0adfc5c-024d-44b1-a451-1905b702a822 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:12.862773 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0adfc5c-024d-44b1-a451-1905b702a822 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:12.887257 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b9ea6-c785-468e-9a7f-da9231e6af97 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:12.890535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d2d01-c7cd-41e4-a27e-7f8edb889b9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:12.890535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d2d01-c7cd-41e4-a27e-7f8edb889b9d service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:12.890535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d2d01-c7cd-41e4-a27e-7f8edb889b9d service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:12.890535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d2d01-c7cd-41e4-a27e-7f8edb889b9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:12.896453 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 994/1970] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:48:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:48:12.897644 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adfc5c-024d-44b1-a451-1905b702a822 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:12.908135 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adfc5c-024d-44b1-a451-1905b702a822 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:12.913419 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad1daa-80f1-4c92-b175-e585bb2a6c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:12.914332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad1daa-80f1-4c92-b175-e585bb2a6c7a admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:12.914714 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad1daa-80f1-4c92-b175-e585bb2a6c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:12.915078 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad1daa-80f1-4c92-b175-e585bb2a6c7a admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:12.915864 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad1daa-80f1-4c92-b175-e585bb2a6c7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:12.918079 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad1daa-80f1-4c92-b175-e585bb2a6c7a admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:12.919281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d2d01-c7cd-41e4-a27e-7f8edb889b9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:12.923165 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d2d01-c7cd-41e4-a27e-7f8edb889b9d service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:12.940036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 977/1971] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 09:48:12.967965 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 995/1972] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:48:12 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:12.969705 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d2d01-c7cd-41e4-a27e-7f8edb889b9d service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:12.986164 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d2d01-c7cd-41e4-a27e-7f8edb889b9d service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:13.124708 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97635161-b2bb-496b-b044-831d70bed08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:13.125519 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97635161-b2bb-496b-b044-831d70bed08b admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:13.125704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97635161-b2bb-496b-b044-831d70bed08b admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:13.125967 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97635161-b2bb-496b-b044-831d70bed08b admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:13.126684 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97635161-b2bb-496b-b044-831d70bed08b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:13.128623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97635161-b2bb-496b-b044-831d70bed08b admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:13.193399 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 978/1973] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:48:12 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:13.194199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad1daa-80f1-4c92-b175-e585bb2a6c7a admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:13.211604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad1daa-80f1-4c92-b175-e585bb2a6c7a admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:13.228903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3234d30-efc3-43eb-9fc5-d744309e8d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:13.230149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3234d30-efc3-43eb-9fc5-d744309e8d91 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:13.230469 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3234d30-efc3-43eb-9fc5-d744309e8d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:13.230743 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3234d30-efc3-43eb-9fc5-d744309e8d91 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:13.231489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3234d30-efc3-43eb-9fc5-d744309e8d91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:13.239001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3234d30-efc3-43eb-9fc5-d744309e8d91 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:13.256534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 996/1974] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:48:13.257444 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97635161-b2bb-496b-b044-831d70bed08b admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:13.277335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97635161-b2bb-496b-b044-831d70bed08b admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:13.301240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d10ae2-8f7a-4680-b7ec-6174f6a180b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:13.301839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d10ae2-8f7a-4680-b7ec-6174f6a180b0 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:13.302124 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d10ae2-8f7a-4680-b7ec-6174f6a180b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:13.302397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d10ae2-8f7a-4680-b7ec-6174f6a180b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:13.303139 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d10ae2-8f7a-4680-b7ec-6174f6a180b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:13.305474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d10ae2-8f7a-4680-b7ec-6174f6a180b0 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:13.337277 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 979/1975] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:48:13.342332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3234d30-efc3-43eb-9fc5-d744309e8d91 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:13.368086 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3234d30-efc3-43eb-9fc5-d744309e8d91 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:13.386588 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3f1a9-8b5e-4b6a-acc3-3c1ff496a07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:13.387524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3f1a9-8b5e-4b6a-acc3-3c1ff496a07e admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:13.387801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3f1a9-8b5e-4b6a-acc3-3c1ff496a07e admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:13.395273 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3f1a9-8b5e-4b6a-acc3-3c1ff496a07e admin admin] PATH_INFO: `/v3/projects/425f3bc385444cd8b0676c7e496322e6/users/9d1a2247a5844199af08b00f4907126a/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:13.476616 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af3f1a9-8b5e-4b6a-acc3-3c1ff496a07e admin admin] RBAC: Authorizing `identity:create_grant(project_id=425f3bc385444cd8b0676c7e496322e6, user_id=9d1a2247a5844199af08b00f4907126a, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:13.479354 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af3f1a9-8b5e-4b6a-acc3-3c1ff496a07e admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:13.521784 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 997/1976] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:13 2022] PUT /identity/v3/projects/425f3bc385444cd8b0676c7e496322e6/users/9d1a2247a5844199af08b00f4907126a/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:13.532821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842188a6-1743-4210-abd7-0d1d80b0a84f None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:13.533493 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842188a6-1743-4210-abd7-0d1d80b0a84f None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:13.533981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842188a6-1743-4210-abd7-0d1d80b0a84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:13.631338 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-842188a6-1743-4210-abd7-0d1d80b0a84f None None] MFA Rules not processed for user `9d1a2247a5844199af08b00f4907126a`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:13.634554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-842188a6-1743-4210-abd7-0d1d80b0a84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:13.769125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3f1a9-8b5e-4b6a-acc3-3c1ff496a07e admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:13.769328 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 980/1977] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:13.900443 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af3f1a9-8b5e-4b6a-acc3-3c1ff496a07e admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:13.922721 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0f13e-9c50-418c-8179-8cd92f0829f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:13.923691 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0f13e-9c50-418c-8179-8cd92f0829f0 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:13.924073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0f13e-9c50-418c-8179-8cd92f0829f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:13.924446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0f13e-9c50-418c-8179-8cd92f0829f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:13.925306 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0f13e-9c50-418c-8179-8cd92f0829f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:13.927354 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0f13e-9c50-418c-8179-8cd92f0829f0 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:13.999101 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 998/1978] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:48:13 2022] POST /identity/v3/projects => generated 387 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:48:14.001481 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842188a6-1743-4210-abd7-0d1d80b0a84f None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:14.011856 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0f13e-9c50-418c-8179-8cd92f0829f0 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:14.013680 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-ade0f13e-9c50-418c-8179-8cd92f0829f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:14.024571 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842188a6-1743-4210-abd7-0d1d80b0a84f None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:14.113222 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0f13e-9c50-418c-8179-8cd92f0829f0 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:14.126976 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17106698-bf09-4bc9-b1e1-6f11b0828803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:14.129588 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17106698-bf09-4bc9-b1e1-6f11b0828803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:14.129866 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17106698-bf09-4bc9-b1e1-6f11b0828803 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:14.132113 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17106698-bf09-4bc9-b1e1-6f11b0828803 admin admin] PATH_INFO: `/v3/projects/9817d732b1e44b828041ac05f0d41e18` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:14.134823 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17106698-bf09-4bc9-b1e1-6f11b0828803 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9817d732b1e44b828041ac05f0d41e18)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:14.137227 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17106698-bf09-4bc9-b1e1-6f11b0828803 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:14.203279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae34a6b-89b2-483f-b5b0-d7202d609879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:14.207067 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae34a6b-89b2-483f-b5b0-d7202d609879 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:14.207574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae34a6b-89b2-483f-b5b0-d7202d609879 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:14.208128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae34a6b-89b2-483f-b5b0-d7202d609879 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:14.210002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae34a6b-89b2-483f-b5b0-d7202d609879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:14.212604 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae34a6b-89b2-483f-b5b0-d7202d609879 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:14.260921 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 981/1979] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:48:13 2022] POST /identity/v3/users => generated 300 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:14.262759 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae34a6b-89b2-483f-b5b0-d7202d609879 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:14.264556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-17106698-bf09-4bc9-b1e1-6f11b0828803 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9817d732b1e44b828041ac05f0d41e18', 'request_id': 'req-17106698-bf09-4bc9-b1e1-6f11b0828803'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:14.278589 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-17106698-bf09-4bc9-b1e1-6f11b0828803 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9817d732b1e44b828041ac05f0d41e18', 'request_id': 'req-17106698-bf09-4bc9-b1e1-6f11b0828803'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:14.290411 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae34a6b-89b2-483f-b5b0-d7202d609879 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:14.291620 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-17106698-bf09-4bc9-b1e1-6f11b0828803 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9817d732b1e44b828041ac05f0d41e18', 'request_id': 'req-17106698-bf09-4bc9-b1e1-6f11b0828803'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:14.318130 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-17106698-bf09-4bc9-b1e1-6f11b0828803 admin admin] The token cache is being invalidate because project 9817d732b1e44b828041ac05f0d41e18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:14.318656 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-17106698-bf09-4bc9-b1e1-6f11b0828803 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:14.319489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-17106698-bf09-4bc9-b1e1-6f11b0828803 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:14.327098 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 999/1980] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:14 2022] DELETE /identity/v3/projects/9817d732b1e44b828041ac05f0d41e18 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:14.328064 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17106698-bf09-4bc9-b1e1-6f11b0828803 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:14.332165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-17106698-bf09-4bc9-b1e1-6f11b0828803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:14.437770 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cda057-7e85-48fd-859f-113f4ea571db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:14.438461 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cda057-7e85-48fd-859f-113f4ea571db admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:14.438735 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cda057-7e85-48fd-859f-113f4ea571db admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:14.439001 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cda057-7e85-48fd-859f-113f4ea571db admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:14.439741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cda057-7e85-48fd-859f-113f4ea571db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:14.441405 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cda057-7e85-48fd-859f-113f4ea571db admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:14.461276 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 982/1981] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:14.463394 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cda057-7e85-48fd-859f-113f4ea571db admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:14.466102 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17106698-bf09-4bc9-b1e1-6f11b0828803 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:14.467021 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-02cda057-7e85-48fd-859f-113f4ea571db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:14.472559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70384c58-cc3f-4569-b92b-fc4ccaaad56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:14.475010 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70384c58-cc3f-4569-b92b-fc4ccaaad56e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:14.475289 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70384c58-cc3f-4569-b92b-fc4ccaaad56e admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:14.475561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70384c58-cc3f-4569-b92b-fc4ccaaad56e admin admin] PATH_INFO: `/v3/users/a0d6261c01fe461680d63fcb461e51bf` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:14.477055 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70384c58-cc3f-4569-b92b-fc4ccaaad56e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0d6261c01fe461680d63fcb461e51bf)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:14.480408 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70384c58-cc3f-4569-b92b-fc4ccaaad56e admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:14.498183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cda057-7e85-48fd-859f-113f4ea571db admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:14.519127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ac4eb-eb58-4318-8772-cc48c1bf826b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:14.520056 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ac4eb-eb58-4318-8772-cc48c1bf826b admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:14.520326 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ac4eb-eb58-4318-8772-cc48c1bf826b admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:14.520596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ac4eb-eb58-4318-8772-cc48c1bf826b admin admin] PATH_INFO: `/v3/projects/bebc96327bfc40c1a14265ac5d5cd826/users/76bc4ae3ba8c4f728bdae15417542df0/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:14.565143 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ac4eb-eb58-4318-8772-cc48c1bf826b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bebc96327bfc40c1a14265ac5d5cd826, user_id=76bc4ae3ba8c4f728bdae15417542df0, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:14.567982 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ac4eb-eb58-4318-8772-cc48c1bf826b admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:14.598455 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 983/1982] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:14 2022] PUT /identity/v3/projects/bebc96327bfc40c1a14265ac5d5cd826/users/76bc4ae3ba8c4f728bdae15417542df0/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:48:14.600785 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ac4eb-eb58-4318-8772-cc48c1bf826b admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:14.627050 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-70384c58-cc3f-4569-b92b-fc4ccaaad56e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0d6261c01fe461680d63fcb461e51bf'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:14.637224 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-70384c58-cc3f-4569-b92b-fc4ccaaad56e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0d6261c01fe461680d63fcb461e51bf'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:14.639259 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-70384c58-cc3f-4569-b92b-fc4ccaaad56e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0d6261c01fe461680d63fcb461e51bf'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:14.655134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-70384c58-cc3f-4569-b92b-fc4ccaaad56e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0d6261c01fe461680d63fcb461e51bf'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:14.656180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-70384c58-cc3f-4569-b92b-fc4ccaaad56e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0d6261c01fe461680d63fcb461e51bf'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:14.707861 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1000/1983] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:14 2022] DELETE /identity/v3/users/a0d6261c01fe461680d63fcb461e51bf => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:14.725604 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ac4eb-eb58-4318-8772-cc48c1bf826b admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:14.853222 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6288c14-9658-41fc-b9c0-bf3f7791cc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:14.854054 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6288c14-9658-41fc-b9c0-bf3f7791cc40 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:14.854460 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6288c14-9658-41fc-b9c0-bf3f7791cc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:14.854849 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6288c14-9658-41fc-b9c0-bf3f7791cc40 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:14.855690 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6288c14-9658-41fc-b9c0-bf3f7791cc40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:14.863840 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6288c14-9658-41fc-b9c0-bf3f7791cc40 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:14.878451 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 984/1984] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:14.882702 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70384c58-cc3f-4569-b92b-fc4ccaaad56e admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:14.885289 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-70384c58-cc3f-4569-b92b-fc4ccaaad56e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:14.909565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70384c58-cc3f-4569-b92b-fc4ccaaad56e admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:14.915347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39ed45-f2cf-46b2-a269-ce40a03176d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:14.917190 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39ed45-f2cf-46b2-a269-ce40a03176d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:14.917626 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39ed45-f2cf-46b2-a269-ce40a03176d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:14.918001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39ed45-f2cf-46b2-a269-ce40a03176d0 admin admin] PATH_INFO: `/v3/projects/bebc96327bfc40c1a14265ac5d5cd826/users/76bc4ae3ba8c4f728bdae15417542df0/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:14.920330 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a39ed45-f2cf-46b2-a269-ce40a03176d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bebc96327bfc40c1a14265ac5d5cd826, user_id=76bc4ae3ba8c4f728bdae15417542df0, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:14.932604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a39ed45-f2cf-46b2-a269-ce40a03176d0 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:14.954221 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1001/1985] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:14 2022] PUT /identity/v3/projects/bebc96327bfc40c1a14265ac5d5cd826/users/76bc4ae3ba8c4f728bdae15417542df0/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:48:14.956291 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc9c06-bb4a-4a90-97e2-4bd2295d8bd0 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:14.956729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc9c06-bb4a-4a90-97e2-4bd2295d8bd0 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:14.957080 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc9c06-bb4a-4a90-97e2-4bd2295d8bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:15.012769 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-aecc9c06-bb4a-4a90-97e2-4bd2295d8bd0 None None] MFA Rules not processed for user `76bc4ae3ba8c4f728bdae15417542df0`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:15.014240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-aecc9c06-bb4a-4a90-97e2-4bd2295d8bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:15.201740 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 985/1986] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:14 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:15.208042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39ed45-f2cf-46b2-a269-ce40a03176d0 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:15.210548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-2a39ed45-f2cf-46b2-a269-ce40a03176d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:15.330667 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39ed45-f2cf-46b2-a269-ce40a03176d0 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:15.338083 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c1037-e8e1-41fe-8695-fe8c30109973 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:15.338803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c1037-e8e1-41fe-8695-fe8c30109973 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:15.339079 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c1037-e8e1-41fe-8695-fe8c30109973 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:15.339347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c1037-e8e1-41fe-8695-fe8c30109973 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:15.362304 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c1037-e8e1-41fe-8695-fe8c30109973 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:15.366052 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c1037-e8e1-41fe-8695-fe8c30109973 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:15.392696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1002/1987] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:48:15.475940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc9c06-bb4a-4a90-97e2-4bd2295d8bd0 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:15.477694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-aecc9c06-bb4a-4a90-97e2-4bd2295d8bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:15.519934 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c1037-e8e1-41fe-8695-fe8c30109973 service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:15.693727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc9c06-bb4a-4a90-97e2-4bd2295d8bd0 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:15.699094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7a028-7196-49e9-a803-c201264684c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:15.699970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7a028-7196-49e9-a803-c201264684c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:15.700244 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7a028-7196-49e9-a803-c201264684c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:15.700545 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7a028-7196-49e9-a803-c201264684c7 admin admin] PATH_INFO: `/v3/projects/20c9d825bb0a48d09e3d278ef395f1a6` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:15.701848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb7a028-7196-49e9-a803-c201264684c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20c9d825bb0a48d09e3d278ef395f1a6)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:15.706615 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb7a028-7196-49e9-a803-c201264684c7 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:15.753231 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c1037-e8e1-41fe-8695-fe8c30109973 service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:15.770243 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e22f1-c30c-4e58-b331-c5b27a890f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:15.774567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e22f1-c30c-4e58-b331-c5b27a890f61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:15.774852 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e22f1-c30c-4e58-b331-c5b27a890f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:15.775266 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e22f1-c30c-4e58-b331-c5b27a890f61 admin admin] PATH_INFO: `/v3/users/9d1a2247a5844199af08b00f4907126a` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:15.781087 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e22f1-c30c-4e58-b331-c5b27a890f61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d1a2247a5844199af08b00f4907126a)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:15.786472 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e22f1-c30c-4e58-b331-c5b27a890f61 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:15.815406 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-ebb7a028-7196-49e9-a803-c201264684c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20c9d825bb0a48d09e3d278ef395f1a6', 'request_id': 'req-ebb7a028-7196-49e9-a803-c201264684c7'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:15.832842 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-ebb7a028-7196-49e9-a803-c201264684c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20c9d825bb0a48d09e3d278ef395f1a6', 'request_id': 'req-ebb7a028-7196-49e9-a803-c201264684c7'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:15.847347 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-ebb7a028-7196-49e9-a803-c201264684c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20c9d825bb0a48d09e3d278ef395f1a6', 'request_id': 'req-ebb7a028-7196-49e9-a803-c201264684c7'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:15.880433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-ebb7a028-7196-49e9-a803-c201264684c7 admin admin] The token cache is being invalidate because project 20c9d825bb0a48d09e3d278ef395f1a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:15.880817 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-ebb7a028-7196-49e9-a803-c201264684c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:15.883132 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-ebb7a028-7196-49e9-a803-c201264684c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:15.887905 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 986/1988] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:15 2022] DELETE /identity/v3/projects/20c9d825bb0a48d09e3d278ef395f1a6 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:15.892919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-629e22f1-c30c-4e58-b331-c5b27a890f61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d1a2247a5844199af08b00f4907126a'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:15.894320 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79db94b-02fd-4fe4-bcc4-2e28e866c067 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:15.894606 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79db94b-02fd-4fe4-bcc4-2e28e866c067 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:15.894875 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79db94b-02fd-4fe4-bcc4-2e28e866c067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:15.912817 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-629e22f1-c30c-4e58-b331-c5b27a890f61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d1a2247a5844199af08b00f4907126a'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:15.913741 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-629e22f1-c30c-4e58-b331-c5b27a890f61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d1a2247a5844199af08b00f4907126a'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:15.932054 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-629e22f1-c30c-4e58-b331-c5b27a890f61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d1a2247a5844199af08b00f4907126a'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:15.932380 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-c79db94b-02fd-4fe4-bcc4-2e28e866c067 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:15.933110 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-629e22f1-c30c-4e58-b331-c5b27a890f61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d1a2247a5844199af08b00f4907126a'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:15.934248 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-c79db94b-02fd-4fe4-bcc4-2e28e866c067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:15.975540 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1003/1989] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:15 2022] DELETE /identity/v3/users/9d1a2247a5844199af08b00f4907126a => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:15.985876 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ca2b5-d458-41b3-8d41-9eda28a02a9b None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:15.991467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ca2b5-d458-41b3-8d41-9eda28a02a9b None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:15.992048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ca2b5-d458-41b3-8d41-9eda28a02a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:16.037816 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-322ca2b5-d458-41b3-8d41-9eda28a02a9b None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:16.040969 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-322ca2b5-d458-41b3-8d41-9eda28a02a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:16.054988 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 987/1990] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:48:16.063547 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d3668-bfcf-4348-b986-de841c9498f2 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:16.063837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d3668-bfcf-4348-b986-de841c9498f2 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:16.064106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d3668-bfcf-4348-b986-de841c9498f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:16.125258 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-199d3668-bfcf-4348-b986-de841c9498f2 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:16.127067 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-199d3668-bfcf-4348-b986-de841c9498f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:16.173748 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1004/1991] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:16.221643 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ca2b5-d458-41b3-8d41-9eda28a02a9b None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:16.226093 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-322ca2b5-d458-41b3-8d41-9eda28a02a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:16.309575 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 988/1992] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:16.314530 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d3668-bfcf-4348-b986-de841c9498f2 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:16.339179 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d3668-bfcf-4348-b986-de841c9498f2 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:16.352143 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3904e4-e2c1-4845-b928-4c2b542c0b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:16.353235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3904e4-e2c1-4845-b928-4c2b542c0b9e admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:16.355033 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3904e4-e2c1-4845-b928-4c2b542c0b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:16.356251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3904e4-e2c1-4845-b928-4c2b542c0b9e admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:16.357329 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3904e4-e2c1-4845-b928-4c2b542c0b9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:48:16.367090 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3904e4-e2c1-4845-b928-4c2b542c0b9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:16.371167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3904e4-e2c1-4845-b928-4c2b542c0b9e admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:16.402153 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 989/1993] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:48:16.404006 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3904e4-e2c1-4845-b928-4c2b542c0b9e admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:16.421687 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3904e4-e2c1-4845-b928-4c2b542c0b9e admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:16.430707 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ca2b5-d458-41b3-8d41-9eda28a02a9b None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:16.436736 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7fb195-655f-4224-9505-57d8fd9bedb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:16.438373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fb195-655f-4224-9505-57d8fd9bedb1 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:16.439396 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fb195-655f-4224-9505-57d8fd9bedb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:16.440391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268fb72-317c-4394-a9a2-fe64a702abd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:16.440895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fb195-655f-4224-9505-57d8fd9bedb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:16.441593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268fb72-317c-4394-a9a2-fe64a702abd0 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:16.442421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7fb195-655f-4224-9505-57d8fd9bedb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:16.442762 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268fb72-317c-4394-a9a2-fe64a702abd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:16.444376 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268fb72-317c-4394-a9a2-fe64a702abd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:16.445779 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7fb195-655f-4224-9505-57d8fd9bedb1 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:16.469406 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268fb72-317c-4394-a9a2-fe64a702abd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:16.474632 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268fb72-317c-4394-a9a2-fe64a702abd0 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:16.514294 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1005/1994] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:48:16.557229 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 990/1995] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:48:16 2022] POST /identity/v3/projects => generated 377 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:16.565847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268fb72-317c-4394-a9a2-fe64a702abd0 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:16.580087 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268fb72-317c-4394-a9a2-fe64a702abd0 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:16.754982 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea682a-30d5-4411-ad14-88fd8edd380d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:16.756612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea682a-30d5-4411-ad14-88fd8edd380d admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:16.758655 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea682a-30d5-4411-ad14-88fd8edd380d admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:16.758990 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea682a-30d5-4411-ad14-88fd8edd380d admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:16.761433 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea682a-30d5-4411-ad14-88fd8edd380d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:16.769205 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea682a-30d5-4411-ad14-88fd8edd380d admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:16.830148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1006/1996] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:48:16 2022] POST /identity/v3/users => generated 289 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:16.831411 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7fb195-655f-4224-9505-57d8fd9bedb1 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:16.848172 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7fb195-655f-4224-9505-57d8fd9bedb1 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:16.853667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88006a1-c385-41de-b4cd-077b9e3839c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:16.854546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88006a1-c385-41de-b4cd-077b9e3839c7 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:16.854987 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88006a1-c385-41de-b4cd-077b9e3839c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:16.855416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88006a1-c385-41de-b4cd-077b9e3839c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:16.856472 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88006a1-c385-41de-b4cd-077b9e3839c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:16.859402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88006a1-c385-41de-b4cd-077b9e3839c7 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:16.874595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 991/1997] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:16.876150 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea682a-30d5-4411-ad14-88fd8edd380d admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:16.888706 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea682a-30d5-4411-ad14-88fd8edd380d admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:16.903475 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42bffa6-850c-4b86-abbb-5baaf44750af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:16.904495 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42bffa6-850c-4b86-abbb-5baaf44750af admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:16.904937 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42bffa6-850c-4b86-abbb-5baaf44750af admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:16.905364 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42bffa6-850c-4b86-abbb-5baaf44750af admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:16.906244 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42bffa6-850c-4b86-abbb-5baaf44750af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:16.908020 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42bffa6-850c-4b86-abbb-5baaf44750af admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:16.938370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88006a1-c385-41de-b4cd-077b9e3839c7 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:16.943163 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1007/1998] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:16.952847 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88006a1-c385-41de-b4cd-077b9e3839c7 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:16.972872 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037f519-0d3d-4b52-8ee8-823fefcc734a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:16.975069 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037f519-0d3d-4b52-8ee8-823fefcc734a admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:16.975435 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037f519-0d3d-4b52-8ee8-823fefcc734a admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:16.976183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037f519-0d3d-4b52-8ee8-823fefcc734a admin admin] PATH_INFO: `/v3/projects/c74347d738b240a5936016e83460444f/users/e9620315f8a74d868949508143d57645/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:17.003675 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e037f519-0d3d-4b52-8ee8-823fefcc734a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c74347d738b240a5936016e83460444f, user_id=e9620315f8a74d868949508143d57645, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:17.007968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e037f519-0d3d-4b52-8ee8-823fefcc734a admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:17.058591 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 992/1999] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:16 2022] PUT /identity/v3/projects/c74347d738b240a5936016e83460444f/users/e9620315f8a74d868949508143d57645/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:17.064201 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8302003-a6d6-4b07-ab22-c317ff8347bc None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:17.066522 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8302003-a6d6-4b07-ab22-c317ff8347bc None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:17.066976 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8302003-a6d6-4b07-ab22-c317ff8347bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:17.118066 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-b8302003-a6d6-4b07-ab22-c317ff8347bc None None] MFA Rules not processed for user `e9620315f8a74d868949508143d57645`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:17.120816 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b8302003-a6d6-4b07-ab22-c317ff8347bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:17.240316 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1008/2000] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:17.246489 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037f519-0d3d-4b52-8ee8-823fefcc734a admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:17.278527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8302003-a6d6-4b07-ab22-c317ff8347bc None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:17.280136 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b8302003-a6d6-4b07-ab22-c317ff8347bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:17.393723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037f519-0d3d-4b52-8ee8-823fefcc734a admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:17.404530 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c68c3-5ec9-4870-b55a-4aa9c0f6f209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:17.405335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c68c3-5ec9-4870-b55a-4aa9c0f6f209 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:17.405905 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c68c3-5ec9-4870-b55a-4aa9c0f6f209 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:17.406142 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c68c3-5ec9-4870-b55a-4aa9c0f6f209 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:17.406971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c68c3-5ec9-4870-b55a-4aa9c0f6f209 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:17.411522 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c68c3-5ec9-4870-b55a-4aa9c0f6f209 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:17.423437 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8302003-a6d6-4b07-ab22-c317ff8347bc None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:17.429975 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67577b6c-885f-44ce-8bbc-58c0df7cc778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:17.431240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67577b6c-885f-44ce-8bbc-58c0df7cc778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:17.432610 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67577b6c-885f-44ce-8bbc-58c0df7cc778 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:17.433106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67577b6c-885f-44ce-8bbc-58c0df7cc778 admin admin] PATH_INFO: `/v3/projects/425f3bc385444cd8b0676c7e496322e6` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:17.434976 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67577b6c-885f-44ce-8bbc-58c0df7cc778 admin admin] RBAC: Authorizing `identity:delete_project(project_id=425f3bc385444cd8b0676c7e496322e6)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:17.438089 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67577b6c-885f-44ce-8bbc-58c0df7cc778 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:17.472842 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 993/2001] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:48:17 2022] POST /identity/v3/projects => generated 379 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:17.477179 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c68c3-5ec9-4870-b55a-4aa9c0f6f209 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:17.542342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-67577b6c-885f-44ce-8bbc-58c0df7cc778 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '425f3bc385444cd8b0676c7e496322e6', 'request_id': 'req-67577b6c-885f-44ce-8bbc-58c0df7cc778'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:17.550185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-67577b6c-885f-44ce-8bbc-58c0df7cc778 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '425f3bc385444cd8b0676c7e496322e6', 'request_id': 'req-67577b6c-885f-44ce-8bbc-58c0df7cc778'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:17.557189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-67577b6c-885f-44ce-8bbc-58c0df7cc778 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '425f3bc385444cd8b0676c7e496322e6', 'request_id': 'req-67577b6c-885f-44ce-8bbc-58c0df7cc778'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:17.578212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-67577b6c-885f-44ce-8bbc-58c0df7cc778 admin admin] The token cache is being invalidate because project 425f3bc385444cd8b0676c7e496322e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:17.578761 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-67577b6c-885f-44ce-8bbc-58c0df7cc778 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:17.579517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-67577b6c-885f-44ce-8bbc-58c0df7cc778 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:17.582667 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1009/2002] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:17 2022] DELETE /identity/v3/projects/425f3bc385444cd8b0676c7e496322e6 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:17.585613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67577b6c-885f-44ce-8bbc-58c0df7cc778 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:17.587957 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-67577b6c-885f-44ce-8bbc-58c0df7cc778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:17.679868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67577b6c-885f-44ce-8bbc-58c0df7cc778 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:17.711692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a9008-29fa-4f4f-ab74-0ef5b1a44143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:17.712028 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c68c3-5ec9-4870-b55a-4aa9c0f6f209 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:17.713651 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a9008-29fa-4f4f-ab74-0ef5b1a44143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:17.714171 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a9008-29fa-4f4f-ab74-0ef5b1a44143 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:17.714663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a9008-29fa-4f4f-ab74-0ef5b1a44143 admin admin] PATH_INFO: `/v3/users/76bc4ae3ba8c4f728bdae15417542df0` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:17.716534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a9008-29fa-4f4f-ab74-0ef5b1a44143 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76bc4ae3ba8c4f728bdae15417542df0)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:17.719407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a9008-29fa-4f4f-ab74-0ef5b1a44143 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:17.728339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae2f35-86bc-4eb8-8cea-9ac38ec75404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:17.729124 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae2f35-86bc-4eb8-8cea-9ac38ec75404 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:17.729407 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae2f35-86bc-4eb8-8cea-9ac38ec75404 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:17.729690 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae2f35-86bc-4eb8-8cea-9ac38ec75404 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:17.730379 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fae2f35-86bc-4eb8-8cea-9ac38ec75404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:17.733973 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fae2f35-86bc-4eb8-8cea-9ac38ec75404 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:17.773396 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 994/2003] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:48:17 2022] POST /identity/v3/users => generated 290 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:48:17.775587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae2f35-86bc-4eb8-8cea-9ac38ec75404 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:17.784577 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-4fae2f35-86bc-4eb8-8cea-9ac38ec75404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:17.804414 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae2f35-86bc-4eb8-8cea-9ac38ec75404 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:17.809879 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808bd49-9c95-4696-8e6d-9fadc5556c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:17.810420 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-733a9008-29fa-4f4f-ab74-0ef5b1a44143 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76bc4ae3ba8c4f728bdae15417542df0'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:17.810705 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808bd49-9c95-4696-8e6d-9fadc5556c5b admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:17.810987 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808bd49-9c95-4696-8e6d-9fadc5556c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:17.811259 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808bd49-9c95-4696-8e6d-9fadc5556c5b admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:17.812034 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808bd49-9c95-4696-8e6d-9fadc5556c5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:17.815704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808bd49-9c95-4696-8e6d-9fadc5556c5b admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:17.835974 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-733a9008-29fa-4f4f-ab74-0ef5b1a44143 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76bc4ae3ba8c4f728bdae15417542df0'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:17.836110 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 995/2004] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:17.837087 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808bd49-9c95-4696-8e6d-9fadc5556c5b admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:17.841839 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-733a9008-29fa-4f4f-ab74-0ef5b1a44143 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76bc4ae3ba8c4f728bdae15417542df0'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:17.853589 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-733a9008-29fa-4f4f-ab74-0ef5b1a44143 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76bc4ae3ba8c4f728bdae15417542df0'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:17.854381 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-733a9008-29fa-4f4f-ab74-0ef5b1a44143 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76bc4ae3ba8c4f728bdae15417542df0'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:17.855645 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808bd49-9c95-4696-8e6d-9fadc5556c5b admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:17.868909 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8aba9d-0edf-48e4-8101-003eb2127cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:17.869974 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8aba9d-0edf-48e4-8101-003eb2127cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:17.871657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8aba9d-0edf-48e4-8101-003eb2127cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:17.873457 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8aba9d-0edf-48e4-8101-003eb2127cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:17.875005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8aba9d-0edf-48e4-8101-003eb2127cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:17.877816 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8aba9d-0edf-48e4-8101-003eb2127cf0 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:17.892904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1010/2005] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:17 2022] DELETE /identity/v3/users/76bc4ae3ba8c4f728bdae15417542df0 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:17.897513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 996/2006] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:17.901877 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a9008-29fa-4f4f-ab74-0ef5b1a44143 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:17.904978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-733a9008-29fa-4f4f-ab74-0ef5b1a44143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:18.075706 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a9008-29fa-4f4f-ab74-0ef5b1a44143 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:18.100082 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99dfca8-d922-44d5-b746-46d23869d8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:18.101937 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99dfca8-d922-44d5-b746-46d23869d8f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:18.102039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99dfca8-d922-44d5-b746-46d23869d8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:18.102279 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99dfca8-d922-44d5-b746-46d23869d8f4 admin admin] PATH_INFO: `/v3/projects/f3cb142ebf0f41159c464b3228e951f5/users/168b76d2e6a24b7ca0ac449e68b37d2e/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:18.126028 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99dfca8-d922-44d5-b746-46d23869d8f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3cb142ebf0f41159c464b3228e951f5, user_id=168b76d2e6a24b7ca0ac449e68b37d2e, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:18.129146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99dfca8-d922-44d5-b746-46d23869d8f4 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:18.151434 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1011/2007] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:17 2022] PUT /identity/v3/projects/f3cb142ebf0f41159c464b3228e951f5/users/168b76d2e6a24b7ca0ac449e68b37d2e/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:18.162740 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fc5a7-647b-46ba-8222-a0cbb9cd6bba None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:18.163212 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fc5a7-647b-46ba-8222-a0cbb9cd6bba None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:18.163589 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fc5a7-647b-46ba-8222-a0cbb9cd6bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:18.236086 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-665fc5a7-647b-46ba-8222-a0cbb9cd6bba None None] MFA Rules not processed for user `168b76d2e6a24b7ca0ac449e68b37d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:18.241541 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-665fc5a7-647b-46ba-8222-a0cbb9cd6bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:18.348405 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 997/2008] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:18.349789 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99dfca8-d922-44d5-b746-46d23869d8f4 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:18.477223 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d54233-36dc-4e1f-8217-343410ca70fd None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:18.477513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d54233-36dc-4e1f-8217-343410ca70fd None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:18.477795 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d54233-36dc-4e1f-8217-343410ca70fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:18.489482 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99dfca8-d922-44d5-b746-46d23869d8f4 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:18.499395 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0033c-2f3f-41cf-a8e5-d69f051057b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:18.500341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0033c-2f3f-41cf-a8e5-d69f051057b7 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:18.500766 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0033c-2f3f-41cf-a8e5-d69f051057b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:18.501407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0033c-2f3f-41cf-a8e5-d69f051057b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:18.502781 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0033c-2f3f-41cf-a8e5-d69f051057b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:18.508119 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0033c-2f3f-41cf-a8e5-d69f051057b7 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:18.552888 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-67d54233-36dc-4e1f-8217-343410ca70fd None None] MFA Rules not processed for user `e536207e603644b69ad040b27970b773`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:18.562238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-67d54233-36dc-4e1f-8217-343410ca70fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:18.566628 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1012/2009] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:48:18 2022] POST /identity/v3/projects => generated 375 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:18.570810 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0033c-2f3f-41cf-a8e5-d69f051057b7 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:18.676834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 998/2010] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:18 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:18.677617 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d54233-36dc-4e1f-8217-343410ca70fd None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:18.682706 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-67d54233-36dc-4e1f-8217-343410ca70fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:18.727349 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0033c-2f3f-41cf-a8e5-d69f051057b7 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:18.743246 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d29156-6bdb-4dbc-9d6d-92b701ed72b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:18.743993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d29156-6bdb-4dbc-9d6d-92b701ed72b1 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:18.744527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d29156-6bdb-4dbc-9d6d-92b701ed72b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:18.744527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d29156-6bdb-4dbc-9d6d-92b701ed72b1 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:18.745207 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d29156-6bdb-4dbc-9d6d-92b701ed72b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:18.747994 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d29156-6bdb-4dbc-9d6d-92b701ed72b1 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:18.781294 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1013/2011] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:48:18 2022] POST /identity/v3/users => generated 294 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:18.783903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d29156-6bdb-4dbc-9d6d-92b701ed72b1 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:18.787974 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-54d29156-6bdb-4dbc-9d6d-92b701ed72b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:18.795720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d54233-36dc-4e1f-8217-343410ca70fd None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:18.809325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283cba8d-01b7-43d9-9f1f-ae0ba2086511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:18.811183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cba8d-01b7-43d9-9f1f-ae0ba2086511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:18.811485 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cba8d-01b7-43d9-9f1f-ae0ba2086511 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:18.811802 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cba8d-01b7-43d9-9f1f-ae0ba2086511 admin admin] PATH_INFO: `/v3/projects/bebc96327bfc40c1a14265ac5d5cd826` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:18.813906 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283cba8d-01b7-43d9-9f1f-ae0ba2086511 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bebc96327bfc40c1a14265ac5d5cd826)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:18.816517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283cba8d-01b7-43d9-9f1f-ae0ba2086511 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:18.877498 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-283cba8d-01b7-43d9-9f1f-ae0ba2086511 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bebc96327bfc40c1a14265ac5d5cd826', 'request_id': 'req-283cba8d-01b7-43d9-9f1f-ae0ba2086511'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:18.891765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-283cba8d-01b7-43d9-9f1f-ae0ba2086511 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bebc96327bfc40c1a14265ac5d5cd826', 'request_id': 'req-283cba8d-01b7-43d9-9f1f-ae0ba2086511'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:18.900051 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-283cba8d-01b7-43d9-9f1f-ae0ba2086511 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bebc96327bfc40c1a14265ac5d5cd826', 'request_id': 'req-283cba8d-01b7-43d9-9f1f-ae0ba2086511'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:18.903538 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d29156-6bdb-4dbc-9d6d-92b701ed72b1 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:18.916791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-283cba8d-01b7-43d9-9f1f-ae0ba2086511 admin admin] The token cache is being invalidate because project bebc96327bfc40c1a14265ac5d5cd826 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:18.917422 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-283cba8d-01b7-43d9-9f1f-ae0ba2086511 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:18.919134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-283cba8d-01b7-43d9-9f1f-ae0ba2086511 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:18.921560 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 999/2012] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:18 2022] DELETE /identity/v3/projects/bebc96327bfc40c1a14265ac5d5cd826 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:18.922584 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283cba8d-01b7-43d9-9f1f-ae0ba2086511 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:18.924434 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-283cba8d-01b7-43d9-9f1f-ae0ba2086511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:18.992479 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33bce3f-a5b1-4425-aa78-ad06a12e6e98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:18.993168 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33bce3f-a5b1-4425-aa78-ad06a12e6e98 service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:18.993484 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33bce3f-a5b1-4425-aa78-ad06a12e6e98 service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:18.993852 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33bce3f-a5b1-4425-aa78-ad06a12e6e98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:18.995612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b33bce3f-a5b1-4425-aa78-ad06a12e6e98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:19.005043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283cba8d-01b7-43d9-9f1f-ae0ba2086511 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:19.010849 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859a516-573a-4d3e-b444-186489d537e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:19.011407 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859a516-573a-4d3e-b444-186489d537e3 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:19.011780 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859a516-573a-4d3e-b444-186489d537e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:19.012117 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859a516-573a-4d3e-b444-186489d537e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:19.013051 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0859a516-573a-4d3e-b444-186489d537e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:19.014710 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0859a516-573a-4d3e-b444-186489d537e3 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:19.028879 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1000/2013] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:19.030634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3730b0-c261-4e88-8ab3-1c01f0a714b5 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:19.031022 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3730b0-c261-4e88-8ab3-1c01f0a714b5 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:19.031401 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3730b0-c261-4e88-8ab3-1c01f0a714b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:19.052687 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-6c3730b0-c261-4e88-8ab3-1c01f0a714b5 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:19.054140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-6c3730b0-c261-4e88-8ab3-1c01f0a714b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:19.066455 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1001/2014] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:19.067782 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33bce3f-a5b1-4425-aa78-ad06a12e6e98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:19.069021 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3730b0-c261-4e88-8ab3-1c01f0a714b5 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:19.070052 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33bce3f-a5b1-4425-aa78-ad06a12e6e98 service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:19.080436 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1014/2015] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:48:19.081925 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3730b0-c261-4e88-8ab3-1c01f0a714b5 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:19.082139 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd693d0-a7bd-4ae2-8499-d2e55613df3c None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:19.082551 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd693d0-a7bd-4ae2-8499-d2e55613df3c None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:19.083167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd693d0-a7bd-4ae2-8499-d2e55613df3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:19.087217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5dab5e-f80c-485f-ac9a-7a7a4e8021d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:19.088368 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5dab5e-f80c-485f-ac9a-7a7a4e8021d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:19.088849 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5dab5e-f80c-485f-ac9a-7a7a4e8021d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:19.089478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5dab5e-f80c-485f-ac9a-7a7a4e8021d0 admin admin] PATH_INFO: `/v3/projects/42144cf89e3a43ad9d09cd18a6bad134/users/6514531aa5ce4e9caa62a48df865a422/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:19.110945 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-9fd693d0-a7bd-4ae2-8499-d2e55613df3c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:19.111826 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5dab5e-f80c-485f-ac9a-7a7a4e8021d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42144cf89e3a43ad9d09cd18a6bad134, user_id=6514531aa5ce4e9caa62a48df865a422, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:19.113986 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-9fd693d0-a7bd-4ae2-8499-d2e55613df3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:19.114818 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5dab5e-f80c-485f-ac9a-7a7a4e8021d0 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:19.136149 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1002/2016] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:19 2022] PUT /identity/v3/projects/42144cf89e3a43ad9d09cd18a6bad134/users/6514531aa5ce4e9caa62a48df865a422/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:19.136337 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1015/2017] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:19.137802 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5dab5e-f80c-485f-ac9a-7a7a4e8021d0 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:19.139862 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd693d0-a7bd-4ae2-8499-d2e55613df3c None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:19.156263 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd693d0-a7bd-4ae2-8499-d2e55613df3c None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:19.248840 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5dab5e-f80c-485f-ac9a-7a7a4e8021d0 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:19.260425 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe8f35-f552-48be-a6d7-630a0d72a997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:19.261503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe8f35-f552-48be-a6d7-630a0d72a997 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:19.262164 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe8f35-f552-48be-a6d7-630a0d72a997 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:19.262744 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe8f35-f552-48be-a6d7-630a0d72a997 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:19.263664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe8f35-f552-48be-a6d7-630a0d72a997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:19.274834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe8f35-f552-48be-a6d7-630a0d72a997 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:19.295623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65fa21c-83d0-4eb7-9815-cddc894e8f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:19.297346 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1003/2018] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:19.299159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fa21c-83d0-4eb7-9815-cddc894e8f18 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:19.299460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fa21c-83d0-4eb7-9815-cddc894e8f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:19.299782 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fa21c-83d0-4eb7-9815-cddc894e8f18 admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:19.301942 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe8f35-f552-48be-a6d7-630a0d72a997 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:19.302798 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65fa21c-83d0-4eb7-9815-cddc894e8f18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:48:19.303136 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65fa21c-83d0-4eb7-9815-cddc894e8f18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:19.307935 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65fa21c-83d0-4eb7-9815-cddc894e8f18 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:19.319591 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe8f35-f552-48be-a6d7-630a0d72a997 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:19.327088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b92e7-12b5-49cc-91ed-1199b18b05fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:19.328354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b92e7-12b5-49cc-91ed-1199b18b05fe admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:19.328957 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b92e7-12b5-49cc-91ed-1199b18b05fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:19.329648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b92e7-12b5-49cc-91ed-1199b18b05fe admin admin] PATH_INFO: `/v3/projects/42144cf89e3a43ad9d09cd18a6bad134/users/6514531aa5ce4e9caa62a48df865a422/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:19.333905 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b92e7-12b5-49cc-91ed-1199b18b05fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=42144cf89e3a43ad9d09cd18a6bad134, user_id=6514531aa5ce4e9caa62a48df865a422, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:19.336368 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1016/2019] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:48:19.339982 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65fa21c-83d0-4eb7-9815-cddc894e8f18 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:19.342637 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b92e7-12b5-49cc-91ed-1199b18b05fe admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:19.365192 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65fa21c-83d0-4eb7-9815-cddc894e8f18 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:19.379403 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1004/2020] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:19 2022] PUT /identity/v3/projects/42144cf89e3a43ad9d09cd18a6bad134/users/6514531aa5ce4e9caa62a48df865a422/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:48:19.381397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ad5e1-20ea-462e-9a6a-5d0155f86e17 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:19.381684 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ad5e1-20ea-462e-9a6a-5d0155f86e17 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:19.382189 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ad5e1-20ea-462e-9a6a-5d0155f86e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:19.448397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-0e9ad5e1-20ea-462e-9a6a-5d0155f86e17 None None] MFA Rules not processed for user `6514531aa5ce4e9caa62a48df865a422`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:19.449658 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-0e9ad5e1-20ea-462e-9a6a-5d0155f86e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:19.526814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d440c6-9290-4213-a379-d5bf5aa1abfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:19.532024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d440c6-9290-4213-a379-d5bf5aa1abfd admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:19.532371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d440c6-9290-4213-a379-d5bf5aa1abfd admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:19.532656 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d440c6-9290-4213-a379-d5bf5aa1abfd admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:19.533413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d440c6-9290-4213-a379-d5bf5aa1abfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:19.535358 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d440c6-9290-4213-a379-d5bf5aa1abfd admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:19.635231 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1017/2021] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:48:19 2022] POST /identity/v3/projects => generated 381 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:48:19.636147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d440c6-9290-4213-a379-d5bf5aa1abfd admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:19.677820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1005/2022] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:19 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:19.690440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ad5e1-20ea-462e-9a6a-5d0155f86e17 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:19.692194 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-0e9ad5e1-20ea-462e-9a6a-5d0155f86e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:19.790733 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d440c6-9290-4213-a379-d5bf5aa1abfd admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:19.798420 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb7381-9b9c-41d3-a855-53f148311bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:19.800001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb7381-9b9c-41d3-a855-53f148311bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:19.800520 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb7381-9b9c-41d3-a855-53f148311bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:19.801038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb7381-9b9c-41d3-a855-53f148311bf3 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:19.803986 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb7381-9b9c-41d3-a855-53f148311bf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:19.806787 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb7381-9b9c-41d3-a855-53f148311bf3 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:19.808595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ad5e1-20ea-462e-9a6a-5d0155f86e17 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:19.845637 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1018/2023] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:48:19 2022] POST /identity/v3/users => generated 291 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:19.850119 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb7381-9b9c-41d3-a855-53f148311bf3 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:19.867384 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb7381-9b9c-41d3-a855-53f148311bf3 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:19.874450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10917e2f-0441-46e9-84a0-f98623e31d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:19.875859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10917e2f-0441-46e9-84a0-f98623e31d81 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:19.876213 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10917e2f-0441-46e9-84a0-f98623e31d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:19.876552 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10917e2f-0441-46e9-84a0-f98623e31d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:19.877390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10917e2f-0441-46e9-84a0-f98623e31d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:19.879096 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10917e2f-0441-46e9-84a0-f98623e31d81 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:19.897275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1019/2024] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:19.899713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10917e2f-0441-46e9-84a0-f98623e31d81 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:19.905174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6257737-51d8-47ce-8f6b-49eda98bc458 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:19.906273 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6257737-51d8-47ce-8f6b-49eda98bc458 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:19.907938 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6257737-51d8-47ce-8f6b-49eda98bc458 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:19.908399 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6257737-51d8-47ce-8f6b-49eda98bc458 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:19.911780 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10917e2f-0441-46e9-84a0-f98623e31d81 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:19.917042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9f354-bc15-48b0-92b9-1d39007875f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:19.918207 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9f354-bc15-48b0-92b9-1d39007875f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:19.918554 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9f354-bc15-48b0-92b9-1d39007875f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:19.918891 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9f354-bc15-48b0-92b9-1d39007875f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:19.919795 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9f354-bc15-48b0-92b9-1d39007875f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:19.921709 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9f354-bc15-48b0-92b9-1d39007875f3 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:19.937521 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1020/2025] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:19.944006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9f354-bc15-48b0-92b9-1d39007875f3 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:19.963543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9f354-bc15-48b0-92b9-1d39007875f3 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:19.974826 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cab187-2f0a-46f4-a6a2-437c588bde5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:19.975777 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cab187-2f0a-46f4-a6a2-437c588bde5d admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:19.976053 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cab187-2f0a-46f4-a6a2-437c588bde5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:19.976712 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cab187-2f0a-46f4-a6a2-437c588bde5d admin admin] PATH_INFO: `/v3/projects/be1e51418b164327a27c05b9a285e7b3/users/316331f4766845a2845eeee3f5d568b2/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:20.006563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cab187-2f0a-46f4-a6a2-437c588bde5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=be1e51418b164327a27c05b9a285e7b3, user_id=316331f4766845a2845eeee3f5d568b2, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:20.010121 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cab187-2f0a-46f4-a6a2-437c588bde5d admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:20.040491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1021/2026] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:19 2022] PUT /identity/v3/projects/be1e51418b164327a27c05b9a285e7b3/users/316331f4766845a2845eeee3f5d568b2/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:20.049759 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6257737-51d8-47ce-8f6b-49eda98bc458 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:20.049759 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6257737-51d8-47ce-8f6b-49eda98bc458 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:20.052106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bd87a-35c5-46a2-993d-0e832f55861b None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:20.052500 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bd87a-35c5-46a2-993d-0e832f55861b None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:20.052647 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bd87a-35c5-46a2-993d-0e832f55861b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:20.062525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1006/2027] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:19 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:48:20.101890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-0f5bd87a-35c5-46a2-993d-0e832f55861b None None] MFA Rules not processed for user `316331f4766845a2845eeee3f5d568b2`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:20.103107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-0f5bd87a-35c5-46a2-993d-0e832f55861b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:20.130298 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6257737-51d8-47ce-8f6b-49eda98bc458 service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:20.133853 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f6257737-51d8-47ce-8f6b-49eda98bc458 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:20.151017 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6257737-51d8-47ce-8f6b-49eda98bc458 service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:20.215366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1022/2028] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:20.217993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5bd87a-35c5-46a2-993d-0e832f55861b None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:20.219466 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b46dd8-5681-4235-abe0-c65b3ec1534c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:20.220140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b46dd8-5681-4235-abe0-c65b3ec1534c service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:20.220917 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b46dd8-5681-4235-abe0-c65b3ec1534c service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:20.221129 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b46dd8-5681-4235-abe0-c65b3ec1534c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:20.226926 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-b9b46dd8-5681-4235-abe0-c65b3ec1534c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:20.327098 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b46dd8-5681-4235-abe0-c65b3ec1534c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:20.329002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b46dd8-5681-4235-abe0-c65b3ec1534c service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:20.343467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1007/2029] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 09:48:20.420236 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5bd87a-35c5-46a2-993d-0e832f55861b None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:20.441414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8a798-eb12-44fb-bfa4-fbb97e5c0c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:20.441414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8a798-eb12-44fb-bfa4-fbb97e5c0c58 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:20.441414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8a798-eb12-44fb-bfa4-fbb97e5c0c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:20.441414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8a798-eb12-44fb-bfa4-fbb97e5c0c58 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:20.441784 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8a798-eb12-44fb-bfa4-fbb97e5c0c58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:20.461214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8a798-eb12-44fb-bfa4-fbb97e5c0c58 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:20.535643 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1023/2030] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:48:20 2022] POST /identity/v3/projects => generated 377 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:20.539946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b46dd8-5681-4235-abe0-c65b3ec1534c service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:20.551750 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b46dd8-5681-4235-abe0-c65b3ec1534c service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:20.667672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6dedd-7fbf-4347-b964-0415f6e0eb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:20.670572 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6dedd-7fbf-4347-b964-0415f6e0eb17 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:20.674351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6dedd-7fbf-4347-b964-0415f6e0eb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:20.674858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6dedd-7fbf-4347-b964-0415f6e0eb17 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:20.676872 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6dedd-7fbf-4347-b964-0415f6e0eb17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:20.683034 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6dedd-7fbf-4347-b964-0415f6e0eb17 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:20.734115 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1008/2031] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:48:20 2022] POST /identity/v3/users => generated 295 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:20.738480 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8a798-eb12-44fb-bfa4-fbb97e5c0c58 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:20.750258 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8a798-eb12-44fb-bfa4-fbb97e5c0c58 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:20.756454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02622bb5-552d-4ea8-86b8-d63798624b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:20.757215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02622bb5-552d-4ea8-86b8-d63798624b8a admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:20.757500 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02622bb5-552d-4ea8-86b8-d63798624b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:20.757780 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02622bb5-552d-4ea8-86b8-d63798624b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:20.758552 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02622bb5-552d-4ea8-86b8-d63798624b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:20.760346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02622bb5-552d-4ea8-86b8-d63798624b8a admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:20.798987 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6dedd-7fbf-4347-b964-0415f6e0eb17 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:20.799430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1024/2032] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:20.815120 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6dedd-7fbf-4347-b964-0415f6e0eb17 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:20.828965 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc645d-3b0c-4f43-a4f9-424ff1f1efe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:20.830963 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc645d-3b0c-4f43-a4f9-424ff1f1efe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:20.834452 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc645d-3b0c-4f43-a4f9-424ff1f1efe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:20.834881 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc645d-3b0c-4f43-a4f9-424ff1f1efe5 admin admin] PATH_INFO: `/v3/projects/c29eb05f95f6426f8c9138c6ba7f41f7/users/7718cb0212ae43268ce9a6b498713d05/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:20.872809 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc645d-3b0c-4f43-a4f9-424ff1f1efe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29eb05f95f6426f8c9138c6ba7f41f7, user_id=7718cb0212ae43268ce9a6b498713d05, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:20.885005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc645d-3b0c-4f43-a4f9-424ff1f1efe5 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:20.918737 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1009/2033] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:20 2022] PUT /identity/v3/projects/c29eb05f95f6426f8c9138c6ba7f41f7/users/7718cb0212ae43268ce9a6b498713d05/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:48:20.918879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02622bb5-552d-4ea8-86b8-d63798624b8a admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:20.930073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02622bb5-552d-4ea8-86b8-d63798624b8a admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:21.138489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48561b19-d205-4f72-bdf6-6e0dfdc1e176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:21.139483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48561b19-d205-4f72-bdf6-6e0dfdc1e176 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:21.140106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48561b19-d205-4f72-bdf6-6e0dfdc1e176 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:21.141179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48561b19-d205-4f72-bdf6-6e0dfdc1e176 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:21.143089 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48561b19-d205-4f72-bdf6-6e0dfdc1e176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:21.145638 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48561b19-d205-4f72-bdf6-6e0dfdc1e176 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:21.172905 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc645d-3b0c-4f43-a4f9-424ff1f1efe5 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:21.175154 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1025/2034] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:21.185468 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc645d-3b0c-4f43-a4f9-424ff1f1efe5 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:21.195514 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42478922-dbfb-4290-88cf-32a22428b8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:21.196585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42478922-dbfb-4290-88cf-32a22428b8ec admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:21.196960 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42478922-dbfb-4290-88cf-32a22428b8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:21.197262 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42478922-dbfb-4290-88cf-32a22428b8ec admin admin] PATH_INFO: `/v3/projects/c29eb05f95f6426f8c9138c6ba7f41f7/users/7718cb0212ae43268ce9a6b498713d05/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:21.217182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42478922-dbfb-4290-88cf-32a22428b8ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29eb05f95f6426f8c9138c6ba7f41f7, user_id=7718cb0212ae43268ce9a6b498713d05, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:21.221346 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42478922-dbfb-4290-88cf-32a22428b8ec admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:21.272708 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1010/2035] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:21 2022] PUT /identity/v3/projects/c29eb05f95f6426f8c9138c6ba7f41f7/users/7718cb0212ae43268ce9a6b498713d05/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:21.277646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b06d7a-5459-4623-bdef-791ed5eed25d None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:21.278732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b06d7a-5459-4623-bdef-791ed5eed25d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:21.278732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b06d7a-5459-4623-bdef-791ed5eed25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:21.338948 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-e4b06d7a-5459-4623-bdef-791ed5eed25d None None] MFA Rules not processed for user `7718cb0212ae43268ce9a6b498713d05`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:21.340911 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e4b06d7a-5459-4623-bdef-791ed5eed25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:21.524385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1026/2036] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:21.532482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42478922-dbfb-4290-88cf-32a22428b8ec admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:21.646099 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42478922-dbfb-4290-88cf-32a22428b8ec admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:21.650981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c09fa-5eb5-4456-b9ae-e8f213fecb26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:21.651671 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c09fa-5eb5-4456-b9ae-e8f213fecb26 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:21.651955 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c09fa-5eb5-4456-b9ae-e8f213fecb26 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:21.652630 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c09fa-5eb5-4456-b9ae-e8f213fecb26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:21.665544 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c09fa-5eb5-4456-b9ae-e8f213fecb26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:21.667051 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c09fa-5eb5-4456-b9ae-e8f213fecb26 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:21.693970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1011/2037] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:48:22.135293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b06d7a-5459-4623-bdef-791ed5eed25d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:22.146166 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b06d7a-5459-4623-bdef-791ed5eed25d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:22.170537 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab093b0-3580-4101-a871-2ffc551ab0ce tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7718cb0212ae43268ce9a6b498713d05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c29eb05f95f6426f8c9138c6ba7f41f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:22.171769 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab093b0-3580-4101-a871-2ffc551ab0ce tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:22.172514 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab093b0-3580-4101-a871-2ffc551ab0ce tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:22.173271 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab093b0-3580-4101-a871-2ffc551ab0ce tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:22.174775 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab093b0-3580-4101-a871-2ffc551ab0ce tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:48:22.175300 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab093b0-3580-4101-a871-2ffc551ab0ce tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:22.177403 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab093b0-3580-4101-a871-2ffc551ab0ce tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:22.198771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1027/2038] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:48:22.204254 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c09fa-5eb5-4456-b9ae-e8f213fecb26 service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:22.213765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c09fa-5eb5-4456-b9ae-e8f213fecb26 service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:22.220637 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0ab49-a169-4909-942e-b8a20f0dc05c tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7718cb0212ae43268ce9a6b498713d05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c29eb05f95f6426f8c9138c6ba7f41f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:22.221409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0ab49-a169-4909-942e-b8a20f0dc05c tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:22.221753 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0ab49-a169-4909-942e-b8a20f0dc05c tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:22.221975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0ab49-a169-4909-942e-b8a20f0dc05c tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:22.222842 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff0ab49-a169-4909-942e-b8a20f0dc05c tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:22.225400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff0ab49-a169-4909-942e-b8a20f0dc05c tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:22.267767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1012/2039] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:48:22 2022] POST /identity/v3/projects => generated 354 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:22.271918 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab093b0-3580-4101-a871-2ffc551ab0ce tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:22.286462 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab093b0-3580-4101-a871-2ffc551ab0ce tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:22.414471 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9038b1-5948-4c0d-be18-1cc433cf4922 tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7718cb0212ae43268ce9a6b498713d05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c29eb05f95f6426f8c9138c6ba7f41f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:22.415445 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9038b1-5948-4c0d-be18-1cc433cf4922 tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:22.415920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9038b1-5948-4c0d-be18-1cc433cf4922 tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:22.416391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9038b1-5948-4c0d-be18-1cc433cf4922 tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:22.417526 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9038b1-5948-4c0d-be18-1cc433cf4922 tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:48:22.417845 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9038b1-5948-4c0d-be18-1cc433cf4922 tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:22.419562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9038b1-5948-4c0d-be18-1cc433cf4922 tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:22.444912 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1028/2040] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:48:22.557777 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0ab49-a169-4909-942e-b8a20f0dc05c tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:22.570892 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0ab49-a169-4909-942e-b8a20f0dc05c tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:22.584824 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2474981-fe2c-4498-8849-30c32b2237d5 tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7718cb0212ae43268ce9a6b498713d05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c29eb05f95f6426f8c9138c6ba7f41f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:22.585974 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2474981-fe2c-4498-8849-30c32b2237d5 tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:22.587402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2474981-fe2c-4498-8849-30c32b2237d5 tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:22.587789 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2474981-fe2c-4498-8849-30c32b2237d5 tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] PATH_INFO: `/v3/projects/e0e5e1b127104319857b33b79b9d5ac7` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:22.590649 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2474981-fe2c-4498-8849-30c32b2237d5 tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e0e5e1b127104319857b33b79b9d5ac7)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:22.593702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2474981-fe2c-4498-8849-30c32b2237d5 tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:22.693211 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-f2474981-fe2c-4498-8849-30c32b2237d5 tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0e5e1b127104319857b33b79b9d5ac7', 'request_id': 'req-f2474981-fe2c-4498-8849-30c32b2237d5'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:22.705515 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-f2474981-fe2c-4498-8849-30c32b2237d5 tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0e5e1b127104319857b33b79b9d5ac7', 'request_id': 'req-f2474981-fe2c-4498-8849-30c32b2237d5'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:22.711917 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-f2474981-fe2c-4498-8849-30c32b2237d5 tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0e5e1b127104319857b33b79b9d5ac7', 'request_id': 'req-f2474981-fe2c-4498-8849-30c32b2237d5'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:22.734713 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-f2474981-fe2c-4498-8849-30c32b2237d5 tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] The token cache is being invalidate because project e0e5e1b127104319857b33b79b9d5ac7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:22.735427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-f2474981-fe2c-4498-8849-30c32b2237d5 tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:22.736293 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-f2474981-fe2c-4498-8849-30c32b2237d5 tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:22.738439 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1013/2041] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:22 2022] DELETE /identity/v3/projects/e0e5e1b127104319857b33b79b9d5ac7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:23.087700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9038b1-5948-4c0d-be18-1cc433cf4922 tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:23.089928 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-6c9038b1-5948-4c0d-be18-1cc433cf4922 tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:23.198882 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9038b1-5948-4c0d-be18-1cc433cf4922 tempest-VolumeQuotaClassesTest-64701339 tempest-VolumeQuotaClassesTest-64701339-project-admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:23.284938 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e8840-28dd-4896-a3dc-1f665b941cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:23.285851 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e8840-28dd-4896-a3dc-1f665b941cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:23.286128 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e8840-28dd-4896-a3dc-1f665b941cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:23.286397 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e8840-28dd-4896-a3dc-1f665b941cd8 admin admin] PATH_INFO: `/v3/users/316331f4766845a2845eeee3f5d568b2` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:23.287645 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e8840-28dd-4896-a3dc-1f665b941cd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=316331f4766845a2845eeee3f5d568b2)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:23.292867 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e8840-28dd-4896-a3dc-1f665b941cd8 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:23.340585 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-5d7e8840-28dd-4896-a3dc-1f665b941cd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '316331f4766845a2845eeee3f5d568b2'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:23.347230 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-5d7e8840-28dd-4896-a3dc-1f665b941cd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '316331f4766845a2845eeee3f5d568b2'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:23.347917 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-5d7e8840-28dd-4896-a3dc-1f665b941cd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '316331f4766845a2845eeee3f5d568b2'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:23.354753 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-5d7e8840-28dd-4896-a3dc-1f665b941cd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '316331f4766845a2845eeee3f5d568b2'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:23.355579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-5d7e8840-28dd-4896-a3dc-1f665b941cd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '316331f4766845a2845eeee3f5d568b2'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:23.373646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1029/2042] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:23 2022] DELETE /identity/v3/users/316331f4766845a2845eeee3f5d568b2 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:23.378787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26f949-d26c-4b06-9d95-c4727f1ddc7b None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:23.379066 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26f949-d26c-4b06-9d95-c4727f1ddc7b None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:23.379365 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26f949-d26c-4b06-9d95-c4727f1ddc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:23.397508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-1b26f949-d26c-4b06-9d95-c4727f1ddc7b None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:23.398736 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-1b26f949-d26c-4b06-9d95-c4727f1ddc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:23.464784 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1014/2043] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:48:23.474956 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e8840-28dd-4896-a3dc-1f665b941cd8 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:23.476525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-5d7e8840-28dd-4896-a3dc-1f665b941cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:23.562413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e8840-28dd-4896-a3dc-1f665b941cd8 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:23.573317 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ff373-9ff6-47ee-b064-a2e8ead645a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:23.574285 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ff373-9ff6-47ee-b064-a2e8ead645a8 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:23.574723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ff373-9ff6-47ee-b064-a2e8ead645a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:23.575157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ff373-9ff6-47ee-b064-a2e8ead645a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:23.595259 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ff373-9ff6-47ee-b064-a2e8ead645a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:23.600048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ff373-9ff6-47ee-b064-a2e8ead645a8 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:23.625872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1030/2044] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:48:24.083045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26f949-d26c-4b06-9d95-c4727f1ddc7b None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:24.084831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-1b26f949-d26c-4b06-9d95-c4727f1ddc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:24.111254 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26f949-d26c-4b06-9d95-c4727f1ddc7b None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:24.126528 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030eb98-91cd-409f-b938-ba1c988918dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:24.128165 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030eb98-91cd-409f-b938-ba1c988918dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:24.131668 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030eb98-91cd-409f-b938-ba1c988918dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:24.132973 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030eb98-91cd-409f-b938-ba1c988918dc admin admin] PATH_INFO: `/v3/projects/be1e51418b164327a27c05b9a285e7b3` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:24.136411 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030eb98-91cd-409f-b938-ba1c988918dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=be1e51418b164327a27c05b9a285e7b3)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:24.138948 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030eb98-91cd-409f-b938-ba1c988918dc admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:24.140119 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ff373-9ff6-47ee-b064-a2e8ead645a8 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:24.141843 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-538ff373-9ff6-47ee-b064-a2e8ead645a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:24.155264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ff373-9ff6-47ee-b064-a2e8ead645a8 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:24.160309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc1897-b293-495a-9d9e-cee9f4805cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:24.161439 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc1897-b293-495a-9d9e-cee9f4805cf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:24.162130 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc1897-b293-495a-9d9e-cee9f4805cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:24.162700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc1897-b293-495a-9d9e-cee9f4805cf3 admin admin] PATH_INFO: `/v3/users/e9620315f8a74d868949508143d57645` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:24.164121 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc1897-b293-495a-9d9e-cee9f4805cf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9620315f8a74d868949508143d57645)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:24.166289 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc1897-b293-495a-9d9e-cee9f4805cf3 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:24.206169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-f030eb98-91cd-409f-b938-ba1c988918dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be1e51418b164327a27c05b9a285e7b3', 'request_id': 'req-f030eb98-91cd-409f-b938-ba1c988918dc'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:24.215877 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-f030eb98-91cd-409f-b938-ba1c988918dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be1e51418b164327a27c05b9a285e7b3', 'request_id': 'req-f030eb98-91cd-409f-b938-ba1c988918dc'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:24.223453 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-f030eb98-91cd-409f-b938-ba1c988918dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be1e51418b164327a27c05b9a285e7b3', 'request_id': 'req-f030eb98-91cd-409f-b938-ba1c988918dc'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:24.243801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8dbc1897-b293-495a-9d9e-cee9f4805cf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9620315f8a74d868949508143d57645'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:24.248010 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-f030eb98-91cd-409f-b938-ba1c988918dc admin admin] The token cache is being invalidate because project be1e51418b164327a27c05b9a285e7b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:24.248308 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-f030eb98-91cd-409f-b938-ba1c988918dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:24.249063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-f030eb98-91cd-409f-b938-ba1c988918dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:24.251917 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8dbc1897-b293-495a-9d9e-cee9f4805cf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9620315f8a74d868949508143d57645'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:24.252848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8dbc1897-b293-495a-9d9e-cee9f4805cf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9620315f8a74d868949508143d57645'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:24.253584 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1015/2045] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:24 2022] DELETE /identity/v3/projects/be1e51418b164327a27c05b9a285e7b3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:24.257667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030eb98-91cd-409f-b938-ba1c988918dc admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:24.259258 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f030eb98-91cd-409f-b938-ba1c988918dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:24.264929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8dbc1897-b293-495a-9d9e-cee9f4805cf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9620315f8a74d868949508143d57645'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:24.265667 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8dbc1897-b293-495a-9d9e-cee9f4805cf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9620315f8a74d868949508143d57645'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:24.284472 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1031/2046] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:24 2022] DELETE /identity/v3/users/e9620315f8a74d868949508143d57645 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:24.290844 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b690bc5-24ff-4b75-be75-a86c4eadc63d None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:24.291208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b690bc5-24ff-4b75-be75-a86c4eadc63d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:24.291562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b690bc5-24ff-4b75-be75-a86c4eadc63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:24.316436 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-7b690bc5-24ff-4b75-be75-a86c4eadc63d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:24.317779 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-7b690bc5-24ff-4b75-be75-a86c4eadc63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:24.343652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030eb98-91cd-409f-b938-ba1c988918dc admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:24.403294 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1032/2047] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:48:24.416549 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b690bc5-24ff-4b75-be75-a86c4eadc63d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:24.418443 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-7b690bc5-24ff-4b75-be75-a86c4eadc63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:24.451727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b7911-abac-4495-a593-c52d18e29560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:24.452892 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b7911-abac-4495-a593-c52d18e29560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:24.453171 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b7911-abac-4495-a593-c52d18e29560 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:24.453440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b7911-abac-4495-a593-c52d18e29560 admin admin] PATH_INFO: `/v3/users/7718cb0212ae43268ce9a6b498713d05` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:24.454703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b7911-abac-4495-a593-c52d18e29560 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7718cb0212ae43268ce9a6b498713d05)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:24.456650 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b7911-abac-4495-a593-c52d18e29560 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:24.477968 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b690bc5-24ff-4b75-be75-a86c4eadc63d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:24.482182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520b4af-0df8-4721-b44c-f45f235f0b7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:24.482851 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520b4af-0df8-4721-b44c-f45f235f0b7f service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:24.483456 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520b4af-0df8-4721-b44c-f45f235f0b7f service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:24.483730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520b4af-0df8-4721-b44c-f45f235f0b7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:24.494536 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520b4af-0df8-4721-b44c-f45f235f0b7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:24.496146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520b4af-0df8-4721-b44c-f45f235f0b7f service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:24.508733 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1033/2048] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 09:48:24.517713 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-2c2b7911-abac-4495-a593-c52d18e29560 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7718cb0212ae43268ce9a6b498713d05'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:24.525142 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-2c2b7911-abac-4495-a593-c52d18e29560 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7718cb0212ae43268ce9a6b498713d05'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:24.535536 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-2c2b7911-abac-4495-a593-c52d18e29560 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7718cb0212ae43268ce9a6b498713d05'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:24.536129 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-2c2b7911-abac-4495-a593-c52d18e29560 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7718cb0212ae43268ce9a6b498713d05'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:24.536923 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-2c2b7911-abac-4495-a593-c52d18e29560 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7718cb0212ae43268ce9a6b498713d05'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:24.559490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1016/2049] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:24 2022] DELETE /identity/v3/users/7718cb0212ae43268ce9a6b498713d05 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:24.802731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520b4af-0df8-4721-b44c-f45f235f0b7f service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:24.804140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b520b4af-0df8-4721-b44c-f45f235f0b7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:24.818469 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520b4af-0df8-4721-b44c-f45f235f0b7f service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:24.874326 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b7911-abac-4495-a593-c52d18e29560 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:24.876283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-2c2b7911-abac-4495-a593-c52d18e29560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:24.917009 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2953a-6dfa-4126-8ccd-3afdc6dcfc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:24.918520 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2953a-6dfa-4126-8ccd-3afdc6dcfc8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:24.919181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2953a-6dfa-4126-8ccd-3afdc6dcfc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:24.919918 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2953a-6dfa-4126-8ccd-3afdc6dcfc8a admin admin] PATH_INFO: `/v3/projects/c74347d738b240a5936016e83460444f` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:24.925801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f2953a-6dfa-4126-8ccd-3afdc6dcfc8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c74347d738b240a5936016e83460444f)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:24.928769 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f2953a-6dfa-4126-8ccd-3afdc6dcfc8a admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:25.007218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-c8f2953a-6dfa-4126-8ccd-3afdc6dcfc8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c74347d738b240a5936016e83460444f', 'request_id': 'req-c8f2953a-6dfa-4126-8ccd-3afdc6dcfc8a'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:25.016704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-c8f2953a-6dfa-4126-8ccd-3afdc6dcfc8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c74347d738b240a5936016e83460444f', 'request_id': 'req-c8f2953a-6dfa-4126-8ccd-3afdc6dcfc8a'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:25.017274 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b7911-abac-4495-a593-c52d18e29560 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:25.023992 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-c8f2953a-6dfa-4126-8ccd-3afdc6dcfc8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c74347d738b240a5936016e83460444f', 'request_id': 'req-c8f2953a-6dfa-4126-8ccd-3afdc6dcfc8a'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:25.041231 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-c8f2953a-6dfa-4126-8ccd-3afdc6dcfc8a admin admin] The token cache is being invalidate because project c74347d738b240a5936016e83460444f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:25.041778 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-c8f2953a-6dfa-4126-8ccd-3afdc6dcfc8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:25.042789 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-c8f2953a-6dfa-4126-8ccd-3afdc6dcfc8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:25.048088 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1034/2050] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:24 2022] DELETE /identity/v3/projects/c74347d738b240a5936016e83460444f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:25.051472 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2953a-6dfa-4126-8ccd-3afdc6dcfc8a admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:25.053352 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c8f2953a-6dfa-4126-8ccd-3afdc6dcfc8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:25.127638 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2953a-6dfa-4126-8ccd-3afdc6dcfc8a admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:25.132944 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71674150-d05c-4fa7-b6fa-6b41bf0dd941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:25.133898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71674150-d05c-4fa7-b6fa-6b41bf0dd941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:25.134233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71674150-d05c-4fa7-b6fa-6b41bf0dd941 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:25.134707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71674150-d05c-4fa7-b6fa-6b41bf0dd941 admin admin] PATH_INFO: `/v3/projects/c29eb05f95f6426f8c9138c6ba7f41f7` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:25.135998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e850f-ea2a-46e0-93bb-3684f6999244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:25.136388 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71674150-d05c-4fa7-b6fa-6b41bf0dd941 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c29eb05f95f6426f8c9138c6ba7f41f7)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:25.137120 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e850f-ea2a-46e0-93bb-3684f6999244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:25.137559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e850f-ea2a-46e0-93bb-3684f6999244 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:25.137789 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e850f-ea2a-46e0-93bb-3684f6999244 admin admin] PATH_INFO: `/v3/users/168b76d2e6a24b7ca0ac449e68b37d2e` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:25.139222 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e850f-ea2a-46e0-93bb-3684f6999244 admin admin] RBAC: Authorizing `identity:delete_user(user_id=168b76d2e6a24b7ca0ac449e68b37d2e)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:25.139876 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71674150-d05c-4fa7-b6fa-6b41bf0dd941 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:25.141587 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e850f-ea2a-46e0-93bb-3684f6999244 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:25.234787 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-920e850f-ea2a-46e0-93bb-3684f6999244 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '168b76d2e6a24b7ca0ac449e68b37d2e'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:25.244627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-71674150-d05c-4fa7-b6fa-6b41bf0dd941 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c29eb05f95f6426f8c9138c6ba7f41f7', 'request_id': 'req-71674150-d05c-4fa7-b6fa-6b41bf0dd941'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:25.251083 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-920e850f-ea2a-46e0-93bb-3684f6999244 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '168b76d2e6a24b7ca0ac449e68b37d2e'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:25.253036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-71674150-d05c-4fa7-b6fa-6b41bf0dd941 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c29eb05f95f6426f8c9138c6ba7f41f7', 'request_id': 'req-71674150-d05c-4fa7-b6fa-6b41bf0dd941'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:25.253382 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-920e850f-ea2a-46e0-93bb-3684f6999244 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '168b76d2e6a24b7ca0ac449e68b37d2e'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:25.263103 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-71674150-d05c-4fa7-b6fa-6b41bf0dd941 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c29eb05f95f6426f8c9138c6ba7f41f7', 'request_id': 'req-71674150-d05c-4fa7-b6fa-6b41bf0dd941'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:25.268190 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-920e850f-ea2a-46e0-93bb-3684f6999244 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '168b76d2e6a24b7ca0ac449e68b37d2e'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:25.269394 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-920e850f-ea2a-46e0-93bb-3684f6999244 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '168b76d2e6a24b7ca0ac449e68b37d2e'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:25.307084 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1035/2051] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:25 2022] DELETE /identity/v3/users/168b76d2e6a24b7ca0ac449e68b37d2e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:25.308025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-71674150-d05c-4fa7-b6fa-6b41bf0dd941 admin admin] The token cache is being invalidate because project c29eb05f95f6426f8c9138c6ba7f41f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:25.308324 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-71674150-d05c-4fa7-b6fa-6b41bf0dd941 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:25.310150 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-71674150-d05c-4fa7-b6fa-6b41bf0dd941 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:25.312233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1017/2052] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:24 2022] DELETE /identity/v3/projects/c29eb05f95f6426f8c9138c6ba7f41f7 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:48:25.317629 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f077a-deda-4334-8d08-615f443e6b3f None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:25.318140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f077a-deda-4334-8d08-615f443e6b3f None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:25.318819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f077a-deda-4334-8d08-615f443e6b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:25.348407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-5b2f077a-deda-4334-8d08-615f443e6b3f None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:25.349672 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-5b2f077a-deda-4334-8d08-615f443e6b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:25.483426 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1036/2053] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:25.493737 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cc0e5-fb3d-40a2-ab99-c0111260570c None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:25.494069 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cc0e5-fb3d-40a2-ab99-c0111260570c None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:25.494389 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cc0e5-fb3d-40a2-ab99-c0111260570c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:25.539170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-527cc0e5-fb3d-40a2-ab99-c0111260570c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:25.540435 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-527cc0e5-fb3d-40a2-ab99-c0111260570c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:25.565102 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1018/2054] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:25.569821 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f077a-deda-4334-8d08-615f443e6b3f None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:25.583317 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f077a-deda-4334-8d08-615f443e6b3f None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:25.596317 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e1d91-06d7-4b07-988c-653e14be8a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:25.597293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e1d91-06d7-4b07-988c-653e14be8a20 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:25.598109 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e1d91-06d7-4b07-988c-653e14be8a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:25.598758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e1d91-06d7-4b07-988c-653e14be8a20 admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:25.601529 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e1d91-06d7-4b07-988c-653e14be8a20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:48:25.602976 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e1d91-06d7-4b07-988c-653e14be8a20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:25.608475 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e1d91-06d7-4b07-988c-653e14be8a20 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:25.643439 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1037/2055] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:48:25.647886 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527cc0e5-fb3d-40a2-ab99-c0111260570c None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:25.662052 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527cc0e5-fb3d-40a2-ab99-c0111260570c None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:25.685521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1491a7a-bad7-4a01-868c-0476cae0a523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:25.686503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1491a7a-bad7-4a01-868c-0476cae0a523 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:25.686984 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1491a7a-bad7-4a01-868c-0476cae0a523 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:25.687626 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1491a7a-bad7-4a01-868c-0476cae0a523 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:25.688855 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1491a7a-bad7-4a01-868c-0476cae0a523 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:25.692130 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1491a7a-bad7-4a01-868c-0476cae0a523 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:25.728159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1019/2056] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:48:25 2022] POST /identity/v3/projects => generated 409 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:25.732474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e1d91-06d7-4b07-988c-653e14be8a20 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:25.743341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e1d91-06d7-4b07-988c-653e14be8a20 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:25.832251 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81d829-9893-4108-b298-53565de16532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:25.832251 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81d829-9893-4108-b298-53565de16532 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:25.832251 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81d829-9893-4108-b298-53565de16532 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:25.832251 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81d829-9893-4108-b298-53565de16532 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:25.832770 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81d829-9893-4108-b298-53565de16532 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:25.834898 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81d829-9893-4108-b298-53565de16532 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:25.843384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1491a7a-bad7-4a01-868c-0476cae0a523 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:25.845661 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-b1491a7a-bad7-4a01-868c-0476cae0a523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:25.862179 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1491a7a-bad7-4a01-868c-0476cae0a523 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:25.862418 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1038/2057] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:48:25 2022] POST /identity/v3/users => generated 305 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:25.867330 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81d829-9893-4108-b298-53565de16532 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:25.880320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81d829-9893-4108-b298-53565de16532 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:25.881506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134f86a-7bcf-4aec-b89c-872a84d37cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:25.883071 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134f86a-7bcf-4aec-b89c-872a84d37cc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:25.883508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134f86a-7bcf-4aec-b89c-872a84d37cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:25.883917 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134f86a-7bcf-4aec-b89c-872a84d37cc2 admin admin] PATH_INFO: `/v3/projects/f3cb142ebf0f41159c464b3228e951f5` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:25.885305 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134f86a-7bcf-4aec-b89c-872a84d37cc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3cb142ebf0f41159c464b3228e951f5)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:25.885835 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8e67c-6477-4d25-8c49-bf4a156e3a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:25.886769 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8e67c-6477-4d25-8c49-bf4a156e3a70 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:25.887149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8e67c-6477-4d25-8c49-bf4a156e3a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:25.887424 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8e67c-6477-4d25-8c49-bf4a156e3a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:25.888221 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a8e67c-6477-4d25-8c49-bf4a156e3a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:25.888768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134f86a-7bcf-4aec-b89c-872a84d37cc2 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:25.889912 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a8e67c-6477-4d25-8c49-bf4a156e3a70 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:25.908579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1039/2058] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:25.910099 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8e67c-6477-4d25-8c49-bf4a156e3a70 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:25.925561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8e67c-6477-4d25-8c49-bf4a156e3a70 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:25.932450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e35fc-19b2-495a-8fcc-b5d031265e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:25.933460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e35fc-19b2-495a-8fcc-b5d031265e0a admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:25.933955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e35fc-19b2-495a-8fcc-b5d031265e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:25.934469 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e35fc-19b2-495a-8fcc-b5d031265e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:25.935434 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e35fc-19b2-495a-8fcc-b5d031265e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:25.937088 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e35fc-19b2-495a-8fcc-b5d031265e0a admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:25.952837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1040/2059] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:25.955642 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e35fc-19b2-495a-8fcc-b5d031265e0a admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:25.958940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-9134f86a-7bcf-4aec-b89c-872a84d37cc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3cb142ebf0f41159c464b3228e951f5', 'request_id': 'req-9134f86a-7bcf-4aec-b89c-872a84d37cc2'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:25.966747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-9134f86a-7bcf-4aec-b89c-872a84d37cc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3cb142ebf0f41159c464b3228e951f5', 'request_id': 'req-9134f86a-7bcf-4aec-b89c-872a84d37cc2'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:25.970989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e35fc-19b2-495a-8fcc-b5d031265e0a admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:25.979848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-9134f86a-7bcf-4aec-b89c-872a84d37cc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3cb142ebf0f41159c464b3228e951f5', 'request_id': 'req-9134f86a-7bcf-4aec-b89c-872a84d37cc2'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:25.996477 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-9134f86a-7bcf-4aec-b89c-872a84d37cc2 admin admin] The token cache is being invalidate because project f3cb142ebf0f41159c464b3228e951f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:25.996795 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-9134f86a-7bcf-4aec-b89c-872a84d37cc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:25.997553 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-9134f86a-7bcf-4aec-b89c-872a84d37cc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:26.001001 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1020/2060] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:25 2022] DELETE /identity/v3/projects/f3cb142ebf0f41159c464b3228e951f5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:26.007180 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134f86a-7bcf-4aec-b89c-872a84d37cc2 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:26.008615 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-9134f86a-7bcf-4aec-b89c-872a84d37cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:26.051001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d44bb-80ea-426d-b035-1c4c8c60cbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:26.052793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d44bb-80ea-426d-b035-1c4c8c60cbb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:26.053212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d44bb-80ea-426d-b035-1c4c8c60cbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:26.053768 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d44bb-80ea-426d-b035-1c4c8c60cbb6 admin admin] PATH_INFO: `/v3/projects/2e1f17b52cb74bb486f33e49e0102aa3/users/6185da34cccc427ebc9f5ebc7d73ce13/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:26.071768 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d44bb-80ea-426d-b035-1c4c8c60cbb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e1f17b52cb74bb486f33e49e0102aa3, user_id=6185da34cccc427ebc9f5ebc7d73ce13, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:26.074926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d44bb-80ea-426d-b035-1c4c8c60cbb6 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:26.092676 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1041/2061] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:25 2022] PUT /identity/v3/projects/2e1f17b52cb74bb486f33e49e0102aa3/users/6185da34cccc427ebc9f5ebc7d73ce13/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:26.103199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48ec4b-f132-4183-a09d-bbc8d985f720 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:26.103966 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48ec4b-f132-4183-a09d-bbc8d985f720 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:26.103966 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48ec4b-f132-4183-a09d-bbc8d985f720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:26.106054 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134f86a-7bcf-4aec-b89c-872a84d37cc2 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:26.133796 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-ef48ec4b-f132-4183-a09d-bbc8d985f720 None None] MFA Rules not processed for user `6185da34cccc427ebc9f5ebc7d73ce13`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:26.136242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-ef48ec4b-f132-4183-a09d-bbc8d985f720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:26.165267 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d34764-5a5c-4e2d-8c90-c010e75e72e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:26.166271 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d34764-5a5c-4e2d-8c90-c010e75e72e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:26.166474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d34764-5a5c-4e2d-8c90-c010e75e72e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:26.166719 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d34764-5a5c-4e2d-8c90-c010e75e72e3 admin admin] PATH_INFO: `/v3/users/6514531aa5ce4e9caa62a48df865a422` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:26.168419 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d34764-5a5c-4e2d-8c90-c010e75e72e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6514531aa5ce4e9caa62a48df865a422)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:26.170468 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d34764-5a5c-4e2d-8c90-c010e75e72e3 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:26.217128 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1042/2062] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:26 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:48:26.221616 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef48ec4b-f132-4183-a09d-bbc8d985f720 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:26.223277 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-ef48ec4b-f132-4183-a09d-bbc8d985f720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:26.237476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef48ec4b-f132-4183-a09d-bbc8d985f720 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:26.242031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c7d34764-5a5c-4e2d-8c90-c010e75e72e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6514531aa5ce4e9caa62a48df865a422'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:26.242183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d28711-e8d5-483f-b0b7-a42d2b2d65d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:26.242827 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d28711-e8d5-483f-b0b7-a42d2b2d65d2 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:26.243115 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d28711-e8d5-483f-b0b7-a42d2b2d65d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:26.243558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d28711-e8d5-483f-b0b7-a42d2b2d65d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:26.244258 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d28711-e8d5-483f-b0b7-a42d2b2d65d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:26.247468 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d28711-e8d5-483f-b0b7-a42d2b2d65d2 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:26.253043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c7d34764-5a5c-4e2d-8c90-c010e75e72e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6514531aa5ce4e9caa62a48df865a422'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:26.263683 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c7d34764-5a5c-4e2d-8c90-c010e75e72e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6514531aa5ce4e9caa62a48df865a422'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:26.264966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c7d34764-5a5c-4e2d-8c90-c010e75e72e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6514531aa5ce4e9caa62a48df865a422'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:26.266145 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c7d34764-5a5c-4e2d-8c90-c010e75e72e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6514531aa5ce4e9caa62a48df865a422'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:26.286424 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1043/2063] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:48:26 2022] POST /identity/v3/projects => generated 409 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:26.289788 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d28711-e8d5-483f-b0b7-a42d2b2d65d2 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:26.291390 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1021/2064] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:26 2022] DELETE /identity/v3/users/6514531aa5ce4e9caa62a48df865a422 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:26.294685 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-91d28711-e8d5-483f-b0b7-a42d2b2d65d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:26.362027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d28711-e8d5-483f-b0b7-a42d2b2d65d2 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:26.422141 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea46da2-73d2-4ac5-ba53-032d7aef80d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:26.422935 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea46da2-73d2-4ac5-ba53-032d7aef80d8 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:26.423215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea46da2-73d2-4ac5-ba53-032d7aef80d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:26.423493 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea46da2-73d2-4ac5-ba53-032d7aef80d8 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:26.424188 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea46da2-73d2-4ac5-ba53-032d7aef80d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:26.426213 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea46da2-73d2-4ac5-ba53-032d7aef80d8 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:26.449557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1044/2065] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:48:26 2022] POST /identity/v3/users => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:48:26.454639 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d34764-5a5c-4e2d-8c90-c010e75e72e3 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:26.465047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d34764-5a5c-4e2d-8c90-c010e75e72e3 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:26.469671 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aba1e2-698f-4e02-bcac-08729f4895a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:26.470327 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aba1e2-698f-4e02-bcac-08729f4895a8 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:26.470602 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aba1e2-698f-4e02-bcac-08729f4895a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:26.470876 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aba1e2-698f-4e02-bcac-08729f4895a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:26.471643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aba1e2-698f-4e02-bcac-08729f4895a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:26.473321 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aba1e2-698f-4e02-bcac-08729f4895a8 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:26.483577 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1022/2066] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:26.488510 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea46da2-73d2-4ac5-ba53-032d7aef80d8 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:26.499818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea46da2-73d2-4ac5-ba53-032d7aef80d8 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:26.505522 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306da003-528a-4b82-8d53-2fca51ec4a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:26.506479 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306da003-528a-4b82-8d53-2fca51ec4a29 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:26.506836 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306da003-528a-4b82-8d53-2fca51ec4a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:26.507110 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306da003-528a-4b82-8d53-2fca51ec4a29 admin admin] PATH_INFO: `/v3/projects/8b628c04ec2e424e90fce6dc7fcb97ec/users/bb07d4e904a64a0f98758b42c31e8ece/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:26.528390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306da003-528a-4b82-8d53-2fca51ec4a29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b628c04ec2e424e90fce6dc7fcb97ec, user_id=bb07d4e904a64a0f98758b42c31e8ece, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:26.531439 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306da003-528a-4b82-8d53-2fca51ec4a29 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:26.556715 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1045/2067] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:26 2022] PUT /identity/v3/projects/8b628c04ec2e424e90fce6dc7fcb97ec/users/bb07d4e904a64a0f98758b42c31e8ece/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:26.560275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aba1e2-698f-4e02-bcac-08729f4895a8 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:26.570563 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aba1e2-698f-4e02-bcac-08729f4895a8 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:26.639361 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0e9ba6-c958-4609-8ea6-f04d9392dc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:26.640052 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e9ba6-c958-4609-8ea6-f04d9392dc55 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:26.640332 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e9ba6-c958-4609-8ea6-f04d9392dc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:26.640629 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e9ba6-c958-4609-8ea6-f04d9392dc55 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:26.641399 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e9ba6-c958-4609-8ea6-f04d9392dc55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:26.643005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e9ba6-c958-4609-8ea6-f04d9392dc55 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:26.653545 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1023/2068] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:26.658838 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306da003-528a-4b82-8d53-2fca51ec4a29 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:26.669021 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306da003-528a-4b82-8d53-2fca51ec4a29 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:26.673774 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275eeb09-13db-4536-a63c-d825de2d6be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:26.674798 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275eeb09-13db-4536-a63c-d825de2d6be4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:26.675074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275eeb09-13db-4536-a63c-d825de2d6be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:26.675344 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275eeb09-13db-4536-a63c-d825de2d6be4 admin admin] PATH_INFO: `/v3/projects/8b628c04ec2e424e90fce6dc7fcb97ec/users/bb07d4e904a64a0f98758b42c31e8ece/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:26.677460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275eeb09-13db-4536-a63c-d825de2d6be4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b628c04ec2e424e90fce6dc7fcb97ec, user_id=bb07d4e904a64a0f98758b42c31e8ece, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:26.680227 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275eeb09-13db-4536-a63c-d825de2d6be4 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:26.693472 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1046/2069] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:26 2022] PUT /identity/v3/projects/8b628c04ec2e424e90fce6dc7fcb97ec/users/bb07d4e904a64a0f98758b42c31e8ece/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:26.706020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbbe07-e28d-482d-be3c-1b5961d2717c None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:26.706323 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbbe07-e28d-482d-be3c-1b5961d2717c None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:26.706677 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbbe07-e28d-482d-be3c-1b5961d2717c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:26.739870 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-b5bbbe07-e28d-482d-be3c-1b5961d2717c None None] MFA Rules not processed for user `bb07d4e904a64a0f98758b42c31e8ece`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:26.741404 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-b5bbbe07-e28d-482d-be3c-1b5961d2717c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:26.826955 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1024/2070] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:26 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:26.849133 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275eeb09-13db-4536-a63c-d825de2d6be4 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:26.851177 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-275eeb09-13db-4536-a63c-d825de2d6be4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:26.913326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275eeb09-13db-4536-a63c-d825de2d6be4 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:26.918681 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09837121-9b8a-4da1-a4a7-edc0f5bc3d57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:26.919136 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09837121-9b8a-4da1-a4a7-edc0f5bc3d57 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:26.919415 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09837121-9b8a-4da1-a4a7-edc0f5bc3d57 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:26.919689 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09837121-9b8a-4da1-a4a7-edc0f5bc3d57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:26.931154 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09837121-9b8a-4da1-a4a7-edc0f5bc3d57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:26.932879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09837121-9b8a-4da1-a4a7-edc0f5bc3d57 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:26.943970 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1047/2071] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:26 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:48:26.956790 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbbe07-e28d-482d-be3c-1b5961d2717c None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:26.958925 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-b5bbbe07-e28d-482d-be3c-1b5961d2717c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:27.043308 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbbe07-e28d-482d-be3c-1b5961d2717c None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:27.048108 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e3fb5-74da-4380-bdb6-6ff305b01860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:27.049164 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e3fb5-74da-4380-bdb6-6ff305b01860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:27.049441 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e3fb5-74da-4380-bdb6-6ff305b01860 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:27.049711 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e3fb5-74da-4380-bdb6-6ff305b01860 admin admin] PATH_INFO: `/v3/projects/42144cf89e3a43ad9d09cd18a6bad134` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:27.050910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e3fb5-74da-4380-bdb6-6ff305b01860 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42144cf89e3a43ad9d09cd18a6bad134)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:27.052992 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e3fb5-74da-4380-bdb6-6ff305b01860 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:27.118946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-707e3fb5-74da-4380-bdb6-6ff305b01860 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42144cf89e3a43ad9d09cd18a6bad134', 'request_id': 'req-707e3fb5-74da-4380-bdb6-6ff305b01860'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:27.127445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-707e3fb5-74da-4380-bdb6-6ff305b01860 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42144cf89e3a43ad9d09cd18a6bad134', 'request_id': 'req-707e3fb5-74da-4380-bdb6-6ff305b01860'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:27.134168 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-707e3fb5-74da-4380-bdb6-6ff305b01860 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42144cf89e3a43ad9d09cd18a6bad134', 'request_id': 'req-707e3fb5-74da-4380-bdb6-6ff305b01860'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:27.158234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-707e3fb5-74da-4380-bdb6-6ff305b01860 admin admin] The token cache is being invalidate because project 42144cf89e3a43ad9d09cd18a6bad134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:27.158899 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-707e3fb5-74da-4380-bdb6-6ff305b01860 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:27.160730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-707e3fb5-74da-4380-bdb6-6ff305b01860 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:27.167222 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1025/2072] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:26 2022] DELETE /identity/v3/projects/42144cf89e3a43ad9d09cd18a6bad134 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:27.170363 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4548720-6371-46cb-91ad-25e8982c908f None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:27.170655 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4548720-6371-46cb-91ad-25e8982c908f None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:27.170926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4548720-6371-46cb-91ad-25e8982c908f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:27.199991 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-e4548720-6371-46cb-91ad-25e8982c908f None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:27.202052 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e4548720-6371-46cb-91ad-25e8982c908f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:27.204240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e3fb5-74da-4380-bdb6-6ff305b01860 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:27.205983 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-707e3fb5-74da-4380-bdb6-6ff305b01860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:27.291958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e3fb5-74da-4380-bdb6-6ff305b01860 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:27.303510 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1116cc0-91e7-4294-9212-c59880f5fd70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:27.304194 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1116cc0-91e7-4294-9212-c59880f5fd70 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:27.304473 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1116cc0-91e7-4294-9212-c59880f5fd70 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:27.304772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1116cc0-91e7-4294-9212-c59880f5fd70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:27.307085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a1116cc0-91e7-4294-9212-c59880f5fd70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:27.331155 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1048/2073] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:27.341549 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706c7df-8816-453a-a769-073042d357b0 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:27.341841 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706c7df-8816-453a-a769-073042d357b0 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:27.342446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706c7df-8816-453a-a769-073042d357b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:27.374048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-9706c7df-8816-453a-a769-073042d357b0 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:27.375324 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-9706c7df-8816-453a-a769-073042d357b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:27.406854 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1049/2074] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:27.408153 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1116cc0-91e7-4294-9212-c59880f5fd70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:27.408941 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706c7df-8816-453a-a769-073042d357b0 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:27.410378 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1116cc0-91e7-4294-9212-c59880f5fd70 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:27.423250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1026/2075] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:27 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 09:48:27.434574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706c7df-8816-453a-a769-073042d357b0 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:27.452883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ec318-e68b-4b9f-b432-16ee3d554349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:27.464032 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ec318-e68b-4b9f-b432-16ee3d554349 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:27.464526 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ec318-e68b-4b9f-b432-16ee3d554349 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:27.464846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ec318-e68b-4b9f-b432-16ee3d554349 admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:27.466522 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ec318-e68b-4b9f-b432-16ee3d554349 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:48:27.466846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ec318-e68b-4b9f-b432-16ee3d554349 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:27.471633 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ec318-e68b-4b9f-b432-16ee3d554349 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:27.490918 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1116cc0-91e7-4294-9212-c59880f5fd70 service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:27.491562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1050/2076] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:48:27.515195 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1116cc0-91e7-4294-9212-c59880f5fd70 service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:27.521226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a571121b-cc79-4799-adb9-efab97c1ff4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:27.523241 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571121b-cc79-4799-adb9-efab97c1ff4c admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:27.523690 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571121b-cc79-4799-adb9-efab97c1ff4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:27.523972 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571121b-cc79-4799-adb9-efab97c1ff4c admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:27.524766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a571121b-cc79-4799-adb9-efab97c1ff4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:27.527747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a571121b-cc79-4799-adb9-efab97c1ff4c admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:27.579859 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1027/2077] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:48:27 2022] POST /identity/v3/projects => generated 385 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:27.585049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ec318-e68b-4b9f-b432-16ee3d554349 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:27.601107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ec318-e68b-4b9f-b432-16ee3d554349 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:27.785581 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b48fb-6d33-40e8-8d60-9498e401fc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:27.787588 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b48fb-6d33-40e8-8d60-9498e401fc54 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:27.788071 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b48fb-6d33-40e8-8d60-9498e401fc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:27.788842 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b48fb-6d33-40e8-8d60-9498e401fc54 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:27.790088 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b48fb-6d33-40e8-8d60-9498e401fc54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:27.792456 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b48fb-6d33-40e8-8d60-9498e401fc54 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:27.821521 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1051/2078] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:48:27 2022] POST /identity/v3/users => generated 293 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:27.823270 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a571121b-cc79-4799-adb9-efab97c1ff4c admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:27.838999 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a571121b-cc79-4799-adb9-efab97c1ff4c admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:27.852445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f95f7-bbb2-4f41-b8d6-127020cafdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:27.853508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f95f7-bbb2-4f41-b8d6-127020cafdf8 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:27.853794 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f95f7-bbb2-4f41-b8d6-127020cafdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:27.854254 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f95f7-bbb2-4f41-b8d6-127020cafdf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:27.858897 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f95f7-bbb2-4f41-b8d6-127020cafdf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:27.861771 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f95f7-bbb2-4f41-b8d6-127020cafdf8 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:27.899467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b48fb-6d33-40e8-8d60-9498e401fc54 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:27.901330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1028/2079] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:48:27.922448 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b48fb-6d33-40e8-8d60-9498e401fc54 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:27.938663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331fa136-86b7-4e72-a05b-0e38f7bd628e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:27.939528 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331fa136-86b7-4e72-a05b-0e38f7bd628e admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:27.939809 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331fa136-86b7-4e72-a05b-0e38f7bd628e admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:27.940078 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331fa136-86b7-4e72-a05b-0e38f7bd628e admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:27.940853 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331fa136-86b7-4e72-a05b-0e38f7bd628e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:27.942445 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331fa136-86b7-4e72-a05b-0e38f7bd628e admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:27.967537 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1052/2080] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:48:27.971723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f95f7-bbb2-4f41-b8d6-127020cafdf8 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:27.988996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f95f7-bbb2-4f41-b8d6-127020cafdf8 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:28.017569 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfd6b1-d696-429a-977e-2d9e794e33c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:28.018601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfd6b1-d696-429a-977e-2d9e794e33c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:28.018924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfd6b1-d696-429a-977e-2d9e794e33c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:28.019195 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfd6b1-d696-429a-977e-2d9e794e33c6 admin admin] PATH_INFO: `/v3/projects/8a2cee1545d34e15b56c29eac5ba3843/users/62832c2c1fa54bdaa3f936c19bb54279/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:28.088301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bfd6b1-d696-429a-977e-2d9e794e33c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a2cee1545d34e15b56c29eac5ba3843, user_id=62832c2c1fa54bdaa3f936c19bb54279, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:28.091730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bfd6b1-d696-429a-977e-2d9e794e33c6 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:28.134066 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1029/2081] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:27 2022] PUT /identity/v3/projects/8a2cee1545d34e15b56c29eac5ba3843/users/62832c2c1fa54bdaa3f936c19bb54279/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:28.137022 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8149654-b156-45ef-bdb8-092325211f6e None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:28.137372 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8149654-b156-45ef-bdb8-092325211f6e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:28.137615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8149654-b156-45ef-bdb8-092325211f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:28.178864 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-b8149654-b156-45ef-bdb8-092325211f6e None None] MFA Rules not processed for user `62832c2c1fa54bdaa3f936c19bb54279`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:28.180300 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b8149654-b156-45ef-bdb8-092325211f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:28.285301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1053/2082] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:28.298542 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfd6b1-d696-429a-977e-2d9e794e33c6 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:28.300226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-62bfd6b1-d696-429a-977e-2d9e794e33c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:28.386461 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfd6b1-d696-429a-977e-2d9e794e33c6 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:28.390563 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5236788-f491-44e5-bdc7-d1323573733e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:28.391245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5236788-f491-44e5-bdc7-d1323573733e service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:28.391520 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5236788-f491-44e5-bdc7-d1323573733e service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:28.391791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5236788-f491-44e5-bdc7-d1323573733e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:28.401798 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5236788-f491-44e5-bdc7-d1323573733e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:28.403287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5236788-f491-44e5-bdc7-d1323573733e service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:28.414232 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1030/2083] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:48:29.485439 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8149654-b156-45ef-bdb8-092325211f6e None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:29.487450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b8149654-b156-45ef-bdb8-092325211f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:29.558671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8149654-b156-45ef-bdb8-092325211f6e None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:29.565149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66be7a-2fc9-487f-bf1d-68c658f597bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:29.566287 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66be7a-2fc9-487f-bf1d-68c658f597bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:29.566706 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66be7a-2fc9-487f-bf1d-68c658f597bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:29.567171 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66be7a-2fc9-487f-bf1d-68c658f597bc admin admin] PATH_INFO: `/v3/users/1c6864349e01464db81396b388eec082` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:29.568843 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c66be7a-2fc9-487f-bf1d-68c658f597bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c6864349e01464db81396b388eec082)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:29.571037 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c66be7a-2fc9-487f-bf1d-68c658f597bc admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:29.625541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-4c66be7a-2fc9-487f-bf1d-68c658f597bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c6864349e01464db81396b388eec082'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:29.632679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-4c66be7a-2fc9-487f-bf1d-68c658f597bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c6864349e01464db81396b388eec082'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:29.633689 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-4c66be7a-2fc9-487f-bf1d-68c658f597bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c6864349e01464db81396b388eec082'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:29.646042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-4c66be7a-2fc9-487f-bf1d-68c658f597bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c6864349e01464db81396b388eec082'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:29.646655 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-4c66be7a-2fc9-487f-bf1d-68c658f597bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c6864349e01464db81396b388eec082'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:29.666511 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1054/2084] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:29 2022] DELETE /identity/v3/users/1c6864349e01464db81396b388eec082 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:29.946568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5236788-f491-44e5-bdc7-d1323573733e service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:29.961907 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5236788-f491-44e5-bdc7-d1323573733e service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:30.044771 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05e6ba-7483-4e62-b50d-20bd0c2d03fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:30.045919 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05e6ba-7483-4e62-b50d-20bd0c2d03fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:30.046686 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05e6ba-7483-4e62-b50d-20bd0c2d03fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:30.047185 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05e6ba-7483-4e62-b50d-20bd0c2d03fe admin admin] PATH_INFO: `/v3/projects/0bf380cd9ee343dd892a2628e8a6e95d` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:30.048524 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05e6ba-7483-4e62-b50d-20bd0c2d03fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bf380cd9ee343dd892a2628e8a6e95d)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:30.050651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05e6ba-7483-4e62-b50d-20bd0c2d03fe admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:30.102982 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-7c05e6ba-7483-4e62-b50d-20bd0c2d03fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bf380cd9ee343dd892a2628e8a6e95d', 'request_id': 'req-7c05e6ba-7483-4e62-b50d-20bd0c2d03fe'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:30.110720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-7c05e6ba-7483-4e62-b50d-20bd0c2d03fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bf380cd9ee343dd892a2628e8a6e95d', 'request_id': 'req-7c05e6ba-7483-4e62-b50d-20bd0c2d03fe'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:30.116468 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-7c05e6ba-7483-4e62-b50d-20bd0c2d03fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bf380cd9ee343dd892a2628e8a6e95d', 'request_id': 'req-7c05e6ba-7483-4e62-b50d-20bd0c2d03fe'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:30.142395 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-7c05e6ba-7483-4e62-b50d-20bd0c2d03fe admin admin] The token cache is being invalidate because project 0bf380cd9ee343dd892a2628e8a6e95d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:30.142395 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-7c05e6ba-7483-4e62-b50d-20bd0c2d03fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:30.142962 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-7c05e6ba-7483-4e62-b50d-20bd0c2d03fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:30.148260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1031/2085] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:29 2022] DELETE /identity/v3/projects/0bf380cd9ee343dd892a2628e8a6e95d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:30.149670 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66be7a-2fc9-487f-bf1d-68c658f597bc admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:30.151291 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-4c66be7a-2fc9-487f-bf1d-68c658f597bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:30.165733 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66be7a-2fc9-487f-bf1d-68c658f597bc admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:30.228131 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a97007-a10f-489c-88c2-cb5d2422e729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:30.229305 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a97007-a10f-489c-88c2-cb5d2422e729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:30.229569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a97007-a10f-489c-88c2-cb5d2422e729 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:30.229833 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a97007-a10f-489c-88c2-cb5d2422e729 admin admin] PATH_INFO: `/v3/users/a21c88c82ef24286ba81be360dd95a22` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:30.231798 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a97007-a10f-489c-88c2-cb5d2422e729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a21c88c82ef24286ba81be360dd95a22)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:30.233792 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a97007-a10f-489c-88c2-cb5d2422e729 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:30.286365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-a5a97007-a10f-489c-88c2-cb5d2422e729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a21c88c82ef24286ba81be360dd95a22'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:30.297715 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-a5a97007-a10f-489c-88c2-cb5d2422e729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a21c88c82ef24286ba81be360dd95a22'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:30.299069 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-a5a97007-a10f-489c-88c2-cb5d2422e729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a21c88c82ef24286ba81be360dd95a22'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:30.308164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-a5a97007-a10f-489c-88c2-cb5d2422e729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a21c88c82ef24286ba81be360dd95a22'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:30.308951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-a5a97007-a10f-489c-88c2-cb5d2422e729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a21c88c82ef24286ba81be360dd95a22'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:30.333882 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1055/2086] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:30 2022] DELETE /identity/v3/users/a21c88c82ef24286ba81be360dd95a22 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:30.521436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05e6ba-7483-4e62-b50d-20bd0c2d03fe admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:30.523117 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-7c05e6ba-7483-4e62-b50d-20bd0c2d03fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:30.537223 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05e6ba-7483-4e62-b50d-20bd0c2d03fe admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:30.598053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf3e5e-e44a-4428-bf26-2d74e945f2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:30.598924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf3e5e-e44a-4428-bf26-2d74e945f2e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:30.599201 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf3e5e-e44a-4428-bf26-2d74e945f2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:30.599470 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf3e5e-e44a-4428-bf26-2d74e945f2e6 admin admin] PATH_INFO: `/v3/projects/dffda6ea6b0647828f836cbc3d447751` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:30.601304 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf3e5e-e44a-4428-bf26-2d74e945f2e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dffda6ea6b0647828f836cbc3d447751)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:30.603224 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf3e5e-e44a-4428-bf26-2d74e945f2e6 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:30.658503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-deaf3e5e-e44a-4428-bf26-2d74e945f2e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dffda6ea6b0647828f836cbc3d447751', 'request_id': 'req-deaf3e5e-e44a-4428-bf26-2d74e945f2e6'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:30.664627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-deaf3e5e-e44a-4428-bf26-2d74e945f2e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dffda6ea6b0647828f836cbc3d447751', 'request_id': 'req-deaf3e5e-e44a-4428-bf26-2d74e945f2e6'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:30.670771 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-deaf3e5e-e44a-4428-bf26-2d74e945f2e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dffda6ea6b0647828f836cbc3d447751', 'request_id': 'req-deaf3e5e-e44a-4428-bf26-2d74e945f2e6'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:30.688256 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-deaf3e5e-e44a-4428-bf26-2d74e945f2e6 admin admin] The token cache is being invalidate because project dffda6ea6b0647828f836cbc3d447751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:30.688750 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-deaf3e5e-e44a-4428-bf26-2d74e945f2e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:30.689509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-deaf3e5e-e44a-4428-bf26-2d74e945f2e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:30.691416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1032/2087] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:30 2022] DELETE /identity/v3/projects/dffda6ea6b0647828f836cbc3d447751 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:30.696061 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41f697-09cc-4734-89e1-3f3b29b56f7c None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:30.696545 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41f697-09cc-4734-89e1-3f3b29b56f7c None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:30.696762 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41f697-09cc-4734-89e1-3f3b29b56f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:30.717958 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-ed41f697-09cc-4734-89e1-3f3b29b56f7c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:30.719785 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-ed41f697-09cc-4734-89e1-3f3b29b56f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:30.790175 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1056/2088] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:30.798361 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d91bde-1123-40a0-bb74-ab6306d65d3c None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:30.798657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d91bde-1123-40a0-bb74-ab6306d65d3c None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:30.799146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d91bde-1123-40a0-bb74-ab6306d65d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:30.817428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-16d91bde-1123-40a0-bb74-ab6306d65d3c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:30.818656 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-16d91bde-1123-40a0-bb74-ab6306d65d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:30.827146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1033/2089] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:30.831257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41f697-09cc-4734-89e1-3f3b29b56f7c None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:30.841057 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41f697-09cc-4734-89e1-3f3b29b56f7c None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:30.845871 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97339f-7f1a-45e9-8801-93509ed077f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:30.846528 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97339f-7f1a-45e9-8801-93509ed077f5 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:30.846805 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97339f-7f1a-45e9-8801-93509ed077f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:30.847073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97339f-7f1a-45e9-8801-93509ed077f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:30.847806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df97339f-7f1a-45e9-8801-93509ed077f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:48:30.848112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df97339f-7f1a-45e9-8801-93509ed077f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:30.849793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df97339f-7f1a-45e9-8801-93509ed077f5 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:30.866007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1057/2090] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:48:30.867265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d91bde-1123-40a0-bb74-ab6306d65d3c None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:30.879503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d91bde-1123-40a0-bb74-ab6306d65d3c None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:30.885162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211e3f5-da75-498b-9801-c82ea9268a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:30.885887 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211e3f5-da75-498b-9801-c82ea9268a70 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:30.886164 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211e3f5-da75-498b-9801-c82ea9268a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:30.886434 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211e3f5-da75-498b-9801-c82ea9268a70 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:30.887446 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211e3f5-da75-498b-9801-c82ea9268a70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:30.889190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211e3f5-da75-498b-9801-c82ea9268a70 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:30.931835 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1034/2091] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:48:30 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:30.937039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97339f-7f1a-45e9-8801-93509ed077f5 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:30.950233 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97339f-7f1a-45e9-8801-93509ed077f5 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:31.020180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea490c69-3c24-4a58-b7ab-019c0dbaa86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:31.020924 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea490c69-3c24-4a58-b7ab-019c0dbaa86f admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:31.021208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea490c69-3c24-4a58-b7ab-019c0dbaa86f admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:31.021489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea490c69-3c24-4a58-b7ab-019c0dbaa86f admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:31.022245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea490c69-3c24-4a58-b7ab-019c0dbaa86f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:31.024176 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea490c69-3c24-4a58-b7ab-019c0dbaa86f admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:31.054824 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1058/2092] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:48:30 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:31.058981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211e3f5-da75-498b-9801-c82ea9268a70 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:31.070749 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211e3f5-da75-498b-9801-c82ea9268a70 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:31.078819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8f757-ced3-41fb-a71c-88e91bc1b608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:31.079501 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8f757-ced3-41fb-a71c-88e91bc1b608 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:31.079777 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8f757-ced3-41fb-a71c-88e91bc1b608 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:31.080045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8f757-ced3-41fb-a71c-88e91bc1b608 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:31.080821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c8f757-ced3-41fb-a71c-88e91bc1b608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:31.082502 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c8f757-ced3-41fb-a71c-88e91bc1b608 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:31.095308 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1035/2093] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:31.098324 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea490c69-3c24-4a58-b7ab-019c0dbaa86f admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:31.111430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea490c69-3c24-4a58-b7ab-019c0dbaa86f admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:31.116210 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6bfc5-c33e-4816-926c-d8cd4b45320c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:31.116946 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6bfc5-c33e-4816-926c-d8cd4b45320c admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:31.117222 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6bfc5-c33e-4816-926c-d8cd4b45320c admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:31.117491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6bfc5-c33e-4816-926c-d8cd4b45320c admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:31.118253 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6bfc5-c33e-4816-926c-d8cd4b45320c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:31.119863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6bfc5-c33e-4816-926c-d8cd4b45320c admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:31.129978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1059/2094] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:31.134217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8f757-ced3-41fb-a71c-88e91bc1b608 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:31.146568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8f757-ced3-41fb-a71c-88e91bc1b608 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:31.154637 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486c7d0-3a34-41ce-97d0-49cf300350a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:31.155555 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486c7d0-3a34-41ce-97d0-49cf300350a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:31.155831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486c7d0-3a34-41ce-97d0-49cf300350a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:31.156100 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486c7d0-3a34-41ce-97d0-49cf300350a7 admin admin] PATH_INFO: `/v3/projects/079562ff682e44cba34b43c844369d3d/users/434ae1a406074e379dbef5366e2a8012/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:31.171791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486c7d0-3a34-41ce-97d0-49cf300350a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=079562ff682e44cba34b43c844369d3d, user_id=434ae1a406074e379dbef5366e2a8012, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:31.174555 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486c7d0-3a34-41ce-97d0-49cf300350a7 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:31.189608 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1036/2095] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:31 2022] PUT /identity/v3/projects/079562ff682e44cba34b43c844369d3d/users/434ae1a406074e379dbef5366e2a8012/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:31.201340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5307e7-8b30-45f1-bc66-e957d52af11a None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:31.201615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5307e7-8b30-45f1-bc66-e957d52af11a None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:31.201891 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5307e7-8b30-45f1-bc66-e957d52af11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:31.229493 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-5f5307e7-8b30-45f1-bc66-e957d52af11a None None] MFA Rules not processed for user `434ae1a406074e379dbef5366e2a8012`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:31.230752 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-5f5307e7-8b30-45f1-bc66-e957d52af11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:31.302289 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1060/2096] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:31.313714 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486c7d0-3a34-41ce-97d0-49cf300350a7 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:31.315406 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-8486c7d0-3a34-41ce-97d0-49cf300350a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:31.375250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486c7d0-3a34-41ce-97d0-49cf300350a7 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:31.379010 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9a0ad-ce03-4528-b542-1a0b5073b9e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:31.379741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9a0ad-ce03-4528-b542-1a0b5073b9e5 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:31.380085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9a0ad-ce03-4528-b542-1a0b5073b9e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:31.380441 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9a0ad-ce03-4528-b542-1a0b5073b9e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:31.389047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9a0ad-ce03-4528-b542-1a0b5073b9e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:31.390489 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9a0ad-ce03-4528-b542-1a0b5073b9e5 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:31.399525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1037/2097] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:48:31.418627 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5307e7-8b30-45f1-bc66-e957d52af11a None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:31.478603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5307e7-8b30-45f1-bc66-e957d52af11a None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:31.483269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b369e6-ca3e-49ce-8b65-a38d5e14ec58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:31.484138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b369e6-ca3e-49ce-8b65-a38d5e14ec58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:31.484408 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b369e6-ca3e-49ce-8b65-a38d5e14ec58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:31.484696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b369e6-ca3e-49ce-8b65-a38d5e14ec58 admin admin] PATH_INFO: `/v3/users/434ae1a406074e379dbef5366e2a8012` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:31.486258 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b369e6-ca3e-49ce-8b65-a38d5e14ec58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=434ae1a406074e379dbef5366e2a8012)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:31.488154 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b369e6-ca3e-49ce-8b65-a38d5e14ec58 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:31.544391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-76b369e6-ca3e-49ce-8b65-a38d5e14ec58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '434ae1a406074e379dbef5366e2a8012'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:31.554828 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-76b369e6-ca3e-49ce-8b65-a38d5e14ec58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '434ae1a406074e379dbef5366e2a8012'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:31.555266 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-76b369e6-ca3e-49ce-8b65-a38d5e14ec58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '434ae1a406074e379dbef5366e2a8012'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:31.574229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-76b369e6-ca3e-49ce-8b65-a38d5e14ec58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '434ae1a406074e379dbef5366e2a8012'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:31.575173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-76b369e6-ca3e-49ce-8b65-a38d5e14ec58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '434ae1a406074e379dbef5366e2a8012'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:31.593320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1061/2098] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:31 2022] DELETE /identity/v3/users/434ae1a406074e379dbef5366e2a8012 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:31.598028 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d82354-87d4-4886-aef9-6b96984fb65c None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:31.598342 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d82354-87d4-4886-aef9-6b96984fb65c None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:31.598611 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d82354-87d4-4886-aef9-6b96984fb65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:31.617553 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-46d82354-87d4-4886-aef9-6b96984fb65c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:31.618790 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-46d82354-87d4-4886-aef9-6b96984fb65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:31.677599 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1038/2099] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:31.687834 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b369e6-ca3e-49ce-8b65-a38d5e14ec58 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:31.689338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-76b369e6-ca3e-49ce-8b65-a38d5e14ec58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:31.749449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b369e6-ca3e-49ce-8b65-a38d5e14ec58 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:31.754664 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93644033-2b4c-477e-82dc-df4b5cbc4b58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:31.755412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93644033-2b4c-477e-82dc-df4b5cbc4b58 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:31.755928 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93644033-2b4c-477e-82dc-df4b5cbc4b58 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:31.756559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93644033-2b4c-477e-82dc-df4b5cbc4b58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:31.766732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93644033-2b4c-477e-82dc-df4b5cbc4b58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:31.768274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93644033-2b4c-477e-82dc-df4b5cbc4b58 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:31.778751 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1062/2100] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:48:32.050154 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d82354-87d4-4886-aef9-6b96984fb65c None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:32.051823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-46d82354-87d4-4886-aef9-6b96984fb65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:32.072598 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d82354-87d4-4886-aef9-6b96984fb65c None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:32.080590 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1d5f6-e636-4e16-a751-7f05bb8eb45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:32.081760 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1d5f6-e636-4e16-a751-7f05bb8eb45d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:32.082100 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1d5f6-e636-4e16-a751-7f05bb8eb45d admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:32.082431 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1d5f6-e636-4e16-a751-7f05bb8eb45d admin admin] PATH_INFO: `/v3/projects/079562ff682e44cba34b43c844369d3d` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:32.084156 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1d5f6-e636-4e16-a751-7f05bb8eb45d admin admin] RBAC: Authorizing `identity:delete_project(project_id=079562ff682e44cba34b43c844369d3d)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:32.087561 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1d5f6-e636-4e16-a751-7f05bb8eb45d admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:32.137762 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-41d1d5f6-e636-4e16-a751-7f05bb8eb45d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '079562ff682e44cba34b43c844369d3d', 'request_id': 'req-41d1d5f6-e636-4e16-a751-7f05bb8eb45d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:32.145439 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-41d1d5f6-e636-4e16-a751-7f05bb8eb45d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '079562ff682e44cba34b43c844369d3d', 'request_id': 'req-41d1d5f6-e636-4e16-a751-7f05bb8eb45d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:32.151499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-41d1d5f6-e636-4e16-a751-7f05bb8eb45d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '079562ff682e44cba34b43c844369d3d', 'request_id': 'req-41d1d5f6-e636-4e16-a751-7f05bb8eb45d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:32.166153 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-41d1d5f6-e636-4e16-a751-7f05bb8eb45d admin admin] The token cache is being invalidate because project 079562ff682e44cba34b43c844369d3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:32.166442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-41d1d5f6-e636-4e16-a751-7f05bb8eb45d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:32.167059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-41d1d5f6-e636-4e16-a751-7f05bb8eb45d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:32.172024 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1039/2101] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:32 2022] DELETE /identity/v3/projects/079562ff682e44cba34b43c844369d3d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:32.173781 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96209e-be02-43d1-8d3d-8e52c1ec4bd0 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:32.174061 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96209e-be02-43d1-8d3d-8e52c1ec4bd0 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:32.174334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96209e-be02-43d1-8d3d-8e52c1ec4bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:32.193996 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-5c96209e-be02-43d1-8d3d-8e52c1ec4bd0 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:32.195219 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-5c96209e-be02-43d1-8d3d-8e52c1ec4bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:32.252264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1063/2102] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:32.260197 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718580aa-c9cf-44c8-ad43-2c8ae4c8e1fe None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:32.260479 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718580aa-c9cf-44c8-ad43-2c8ae4c8e1fe None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:32.260791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718580aa-c9cf-44c8-ad43-2c8ae4c8e1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:32.281444 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-718580aa-c9cf-44c8-ad43-2c8ae4c8e1fe None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:32.282687 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-718580aa-c9cf-44c8-ad43-2c8ae4c8e1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:32.291638 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1040/2103] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:32.296047 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96209e-be02-43d1-8d3d-8e52c1ec4bd0 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:32.306267 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96209e-be02-43d1-8d3d-8e52c1ec4bd0 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:32.311295 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6dbd21-1cbb-4a89-af67-d3edc7e2fbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:32.311947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dbd21-1cbb-4a89-af67-d3edc7e2fbcb admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:32.312225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dbd21-1cbb-4a89-af67-d3edc7e2fbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:32.312500 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dbd21-1cbb-4a89-af67-d3edc7e2fbcb admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:32.313269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6dbd21-1cbb-4a89-af67-d3edc7e2fbcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:48:32.313572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6dbd21-1cbb-4a89-af67-d3edc7e2fbcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:32.315177 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6dbd21-1cbb-4a89-af67-d3edc7e2fbcb admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:32.327346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1064/2104] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:48:32.331633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718580aa-c9cf-44c8-ad43-2c8ae4c8e1fe None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:32.341101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718580aa-c9cf-44c8-ad43-2c8ae4c8e1fe None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:32.347547 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe13a9-a9e5-4833-b276-7689e5e1ddf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:32.348212 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe13a9-a9e5-4833-b276-7689e5e1ddf2 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:32.348486 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe13a9-a9e5-4833-b276-7689e5e1ddf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:32.348777 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe13a9-a9e5-4833-b276-7689e5e1ddf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:32.349502 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe13a9-a9e5-4833-b276-7689e5e1ddf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:32.351414 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe13a9-a9e5-4833-b276-7689e5e1ddf2 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:32.383395 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1041/2105] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:48:32 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:32.384711 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6dbd21-1cbb-4a89-af67-d3edc7e2fbcb admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:32.394775 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6dbd21-1cbb-4a89-af67-d3edc7e2fbcb admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:32.452833 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db6b93-b469-451c-95b0-27c7b979df22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:32.453529 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db6b93-b469-451c-95b0-27c7b979df22 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:32.453808 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db6b93-b469-451c-95b0-27c7b979df22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:32.454081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db6b93-b469-451c-95b0-27c7b979df22 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:32.454771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db6b93-b469-451c-95b0-27c7b979df22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:32.456739 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db6b93-b469-451c-95b0-27c7b979df22 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:32.481735 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1065/2106] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:48:32 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:32.483025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe13a9-a9e5-4833-b276-7689e5e1ddf2 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:32.493656 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe13a9-a9e5-4833-b276-7689e5e1ddf2 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:32.498569 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2775c65e-5f5b-45d1-8fe2-3442118ea59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:32.499221 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775c65e-5f5b-45d1-8fe2-3442118ea59c admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:32.499495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775c65e-5f5b-45d1-8fe2-3442118ea59c admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:32.499772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775c65e-5f5b-45d1-8fe2-3442118ea59c admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:32.500526 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2775c65e-5f5b-45d1-8fe2-3442118ea59c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:32.502214 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2775c65e-5f5b-45d1-8fe2-3442118ea59c admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:32.512754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1042/2107] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:32.516802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db6b93-b469-451c-95b0-27c7b979df22 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:32.527002 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db6b93-b469-451c-95b0-27c7b979df22 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:32.531875 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40e3a2-3dad-4e90-a256-7787ca8d691c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:32.532541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40e3a2-3dad-4e90-a256-7787ca8d691c admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:32.532896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40e3a2-3dad-4e90-a256-7787ca8d691c admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:32.533169 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40e3a2-3dad-4e90-a256-7787ca8d691c admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:32.533908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c40e3a2-3dad-4e90-a256-7787ca8d691c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:32.535506 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c40e3a2-3dad-4e90-a256-7787ca8d691c admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:32.545484 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1066/2108] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:32.550117 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2775c65e-5f5b-45d1-8fe2-3442118ea59c admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:32.560908 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2775c65e-5f5b-45d1-8fe2-3442118ea59c admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:32.566677 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2cfbaf-1b0e-401a-9ea2-425a53c68ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:32.567569 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2cfbaf-1b0e-401a-9ea2-425a53c68ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:32.567848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2cfbaf-1b0e-401a-9ea2-425a53c68ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:32.568118 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2cfbaf-1b0e-401a-9ea2-425a53c68ff4 admin admin] PATH_INFO: `/v3/projects/d5f05a4b0e2f496992c70c31b4aa26bc/users/2096c28d3dba4306a6e0059e137b949b/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:32.585433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2cfbaf-1b0e-401a-9ea2-425a53c68ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5f05a4b0e2f496992c70c31b4aa26bc, user_id=2096c28d3dba4306a6e0059e137b949b, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:32.588199 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2cfbaf-1b0e-401a-9ea2-425a53c68ff4 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:32.605284 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1043/2109] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:32 2022] PUT /identity/v3/projects/d5f05a4b0e2f496992c70c31b4aa26bc/users/2096c28d3dba4306a6e0059e137b949b/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:32.616314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a8942-2464-46dc-a8dd-37989f381848 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:32.616606 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a8942-2464-46dc-a8dd-37989f381848 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:32.616910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a8942-2464-46dc-a8dd-37989f381848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:32.644253 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-fc5a8942-2464-46dc-a8dd-37989f381848 None None] MFA Rules not processed for user `2096c28d3dba4306a6e0059e137b949b`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:32.645487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-fc5a8942-2464-46dc-a8dd-37989f381848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:32.709487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1067/2110] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:32.721289 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2cfbaf-1b0e-401a-9ea2-425a53c68ff4 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:32.723343 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-3b2cfbaf-1b0e-401a-9ea2-425a53c68ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:32.785952 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2cfbaf-1b0e-401a-9ea2-425a53c68ff4 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:32.791580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1a4ad-f266-4add-ba97-f37923734ea6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:32.792625 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1a4ad-f266-4add-ba97-f37923734ea6 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:32.793046 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1a4ad-f266-4add-ba97-f37923734ea6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:32.793396 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1a4ad-f266-4add-ba97-f37923734ea6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:32.802862 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1a4ad-f266-4add-ba97-f37923734ea6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:32.804361 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1a4ad-f266-4add-ba97-f37923734ea6 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:32.813689 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1044/2111] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:48:44.235372 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a8942-2464-46dc-a8dd-37989f381848 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:44.236939 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-fc5a8942-2464-46dc-a8dd-37989f381848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:44.301918 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a8942-2464-46dc-a8dd-37989f381848 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:44.306499 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0b5f2-24f4-452d-8ac8-7ef12dd83aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:44.307380 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0b5f2-24f4-452d-8ac8-7ef12dd83aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:44.307625 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0b5f2-24f4-452d-8ac8-7ef12dd83aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:44.307890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0b5f2-24f4-452d-8ac8-7ef12dd83aa8 admin admin] PATH_INFO: `/v3/users/62832c2c1fa54bdaa3f936c19bb54279` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:44.309135 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0b5f2-24f4-452d-8ac8-7ef12dd83aa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62832c2c1fa54bdaa3f936c19bb54279)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:44.311127 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0b5f2-24f4-452d-8ac8-7ef12dd83aa8 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:44.355769 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-54b0b5f2-24f4-452d-8ac8-7ef12dd83aa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62832c2c1fa54bdaa3f936c19bb54279'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:44.362441 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-54b0b5f2-24f4-452d-8ac8-7ef12dd83aa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62832c2c1fa54bdaa3f936c19bb54279'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:44.363079 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-54b0b5f2-24f4-452d-8ac8-7ef12dd83aa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62832c2c1fa54bdaa3f936c19bb54279'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:44.371194 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-54b0b5f2-24f4-452d-8ac8-7ef12dd83aa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62832c2c1fa54bdaa3f936c19bb54279'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:44.371791 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-54b0b5f2-24f4-452d-8ac8-7ef12dd83aa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62832c2c1fa54bdaa3f936c19bb54279'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:44.388432 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1068/2112] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:44 2022] DELETE /identity/v3/users/62832c2c1fa54bdaa3f936c19bb54279 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:44.393075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83081686-12ac-4e79-b766-ea7cb5af7ef6 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:44.393544 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83081686-12ac-4e79-b766-ea7cb5af7ef6 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:44.393634 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83081686-12ac-4e79-b766-ea7cb5af7ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:44.412614 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-83081686-12ac-4e79-b766-ea7cb5af7ef6 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:44.413859 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-83081686-12ac-4e79-b766-ea7cb5af7ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:44.471039 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1045/2113] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:44.485520 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0b5f2-24f4-452d-8ac8-7ef12dd83aa8 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:44.487285 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-54b0b5f2-24f4-452d-8ac8-7ef12dd83aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:44.549040 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0b5f2-24f4-452d-8ac8-7ef12dd83aa8 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:44.553457 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf368f-0741-41eb-b7a3-0e98a6837d6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:44.554124 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf368f-0741-41eb-b7a3-0e98a6837d6a service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:44.554398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf368f-0741-41eb-b7a3-0e98a6837d6a service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:44.555768 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf368f-0741-41eb-b7a3-0e98a6837d6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:44.566227 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf368f-0741-41eb-b7a3-0e98a6837d6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:44.569191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf368f-0741-41eb-b7a3-0e98a6837d6a service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:44.580777 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1069/2114] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:48:44.925690 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83081686-12ac-4e79-b766-ea7cb5af7ef6 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:44.927733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-83081686-12ac-4e79-b766-ea7cb5af7ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:44.943550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83081686-12ac-4e79-b766-ea7cb5af7ef6 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:44.948583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee0a2f-5dc0-4f07-a515-9b8a1c35f253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:44.949590 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee0a2f-5dc0-4f07-a515-9b8a1c35f253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:44.949782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee0a2f-5dc0-4f07-a515-9b8a1c35f253 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:44.950070 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee0a2f-5dc0-4f07-a515-9b8a1c35f253 admin admin] PATH_INFO: `/v3/projects/8a2cee1545d34e15b56c29eac5ba3843` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:44.951351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee0a2f-5dc0-4f07-a515-9b8a1c35f253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a2cee1545d34e15b56c29eac5ba3843)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:44.953296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee0a2f-5dc0-4f07-a515-9b8a1c35f253 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:45.001776 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-dbee0a2f-5dc0-4f07-a515-9b8a1c35f253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a2cee1545d34e15b56c29eac5ba3843', 'request_id': 'req-dbee0a2f-5dc0-4f07-a515-9b8a1c35f253'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:45.008135 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-dbee0a2f-5dc0-4f07-a515-9b8a1c35f253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a2cee1545d34e15b56c29eac5ba3843', 'request_id': 'req-dbee0a2f-5dc0-4f07-a515-9b8a1c35f253'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:45.014057 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-dbee0a2f-5dc0-4f07-a515-9b8a1c35f253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a2cee1545d34e15b56c29eac5ba3843', 'request_id': 'req-dbee0a2f-5dc0-4f07-a515-9b8a1c35f253'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:45.027275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-dbee0a2f-5dc0-4f07-a515-9b8a1c35f253 admin admin] The token cache is being invalidate because project 8a2cee1545d34e15b56c29eac5ba3843 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:45.027555 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-dbee0a2f-5dc0-4f07-a515-9b8a1c35f253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:45.028238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-dbee0a2f-5dc0-4f07-a515-9b8a1c35f253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:45.029893 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1046/2115] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:44 2022] DELETE /identity/v3/projects/8a2cee1545d34e15b56c29eac5ba3843 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:45.036128 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44238f8-c8fc-409b-b98e-3e728a390524 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:45.036486 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44238f8-c8fc-409b-b98e-3e728a390524 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:45.036792 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44238f8-c8fc-409b-b98e-3e728a390524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:45.055868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-d44238f8-c8fc-409b-b98e-3e728a390524 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:45.057123 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-d44238f8-c8fc-409b-b98e-3e728a390524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:45.116913 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1070/2116] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:48:45.125983 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4f3fb-21f7-4348-9252-53fbb0f0d4d7 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:45.126262 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4f3fb-21f7-4348-9252-53fbb0f0d4d7 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:45.126529 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4f3fb-21f7-4348-9252-53fbb0f0d4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:45.146845 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-c0d4f3fb-21f7-4348-9252-53fbb0f0d4d7 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:45.148080 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-c0d4f3fb-21f7-4348-9252-53fbb0f0d4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:45.157667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1047/2117] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:45.162238 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44238f8-c8fc-409b-b98e-3e728a390524 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:45.173978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44238f8-c8fc-409b-b98e-3e728a390524 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:45.179104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411d568-4860-46ac-9e38-d0d02c01f2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:45.179937 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411d568-4860-46ac-9e38-d0d02c01f2c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:45.180181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411d568-4860-46ac-9e38-d0d02c01f2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:45.180477 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411d568-4860-46ac-9e38-d0d02c01f2c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:45.181328 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411d568-4860-46ac-9e38-d0d02c01f2c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:48:45.181794 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411d568-4860-46ac-9e38-d0d02c01f2c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:45.183405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411d568-4860-46ac-9e38-d0d02c01f2c4 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:45.195787 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1071/2118] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:48:45.200411 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4f3fb-21f7-4348-9252-53fbb0f0d4d7 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:45.210168 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4f3fb-21f7-4348-9252-53fbb0f0d4d7 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:45.215537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c72f29-36b0-4822-b205-3169cbbae291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:45.216313 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c72f29-36b0-4822-b205-3169cbbae291 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:45.216834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c72f29-36b0-4822-b205-3169cbbae291 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:45.217111 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c72f29-36b0-4822-b205-3169cbbae291 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:45.217812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c72f29-36b0-4822-b205-3169cbbae291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:45.219737 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c72f29-36b0-4822-b205-3169cbbae291 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:45.246823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1048/2119] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:48:45 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:45.251302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411d568-4860-46ac-9e38-d0d02c01f2c4 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:45.261801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411d568-4860-46ac-9e38-d0d02c01f2c4 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:45.325756 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba7d1f-dc9e-4879-bd26-86b811d2f5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:45.326434 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba7d1f-dc9e-4879-bd26-86b811d2f5c4 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:45.326709 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba7d1f-dc9e-4879-bd26-86b811d2f5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:45.326975 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba7d1f-dc9e-4879-bd26-86b811d2f5c4 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:45.327660 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba7d1f-dc9e-4879-bd26-86b811d2f5c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:45.329612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba7d1f-dc9e-4879-bd26-86b811d2f5c4 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:45.348523 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1072/2120] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:48:45 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:45.352767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c72f29-36b0-4822-b205-3169cbbae291 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:45.363512 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c72f29-36b0-4822-b205-3169cbbae291 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:45.368115 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee3201-26d7-43e0-bf61-0baefdf4837b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:45.368810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee3201-26d7-43e0-bf61-0baefdf4837b admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:45.369088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee3201-26d7-43e0-bf61-0baefdf4837b admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:45.369358 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee3201-26d7-43e0-bf61-0baefdf4837b admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:45.370094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee3201-26d7-43e0-bf61-0baefdf4837b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:45.371697 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee3201-26d7-43e0-bf61-0baefdf4837b admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:45.381800 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1049/2121] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:45.386262 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba7d1f-dc9e-4879-bd26-86b811d2f5c4 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:45.398529 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba7d1f-dc9e-4879-bd26-86b811d2f5c4 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:45.404969 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516241ce-acf2-45e3-9bb2-a5e7842f8604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:45.405620 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516241ce-acf2-45e3-9bb2-a5e7842f8604 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:45.405896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516241ce-acf2-45e3-9bb2-a5e7842f8604 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:45.406157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516241ce-acf2-45e3-9bb2-a5e7842f8604 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:45.406878 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516241ce-acf2-45e3-9bb2-a5e7842f8604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:45.408811 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516241ce-acf2-45e3-9bb2-a5e7842f8604 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:45.418880 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1073/2122] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:48:45.424100 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee3201-26d7-43e0-bf61-0baefdf4837b admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:45.435700 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee3201-26d7-43e0-bf61-0baefdf4837b admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:45.443224 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973040ef-f31b-44e4-bdfb-003f493192fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:45.444740 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973040ef-f31b-44e4-bdfb-003f493192fc admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:45.445237 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973040ef-f31b-44e4-bdfb-003f493192fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:45.445721 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973040ef-f31b-44e4-bdfb-003f493192fc admin admin] PATH_INFO: `/v3/projects/b2ffa13056434f0a8fcd41b2470671bc/users/32a253ad3b764e869aa979d7ba1df6ad/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:45.465270 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973040ef-f31b-44e4-bdfb-003f493192fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2ffa13056434f0a8fcd41b2470671bc, user_id=32a253ad3b764e869aa979d7ba1df6ad, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:45.469461 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973040ef-f31b-44e4-bdfb-003f493192fc admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:45.488335 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1050/2123] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:45 2022] PUT /identity/v3/projects/b2ffa13056434f0a8fcd41b2470671bc/users/32a253ad3b764e869aa979d7ba1df6ad/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:45.498702 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a408b81-95b4-4be6-bbd3-a00334cee100 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:45.498994 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a408b81-95b4-4be6-bbd3-a00334cee100 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:45.499267 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a408b81-95b4-4be6-bbd3-a00334cee100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:45.526372 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-7a408b81-95b4-4be6-bbd3-a00334cee100 None None] MFA Rules not processed for user `32a253ad3b764e869aa979d7ba1df6ad`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:45.527523 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-7a408b81-95b4-4be6-bbd3-a00334cee100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:45.593060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1074/2124] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:45 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:45.605800 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973040ef-f31b-44e4-bdfb-003f493192fc admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:45.607533 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-973040ef-f31b-44e4-bdfb-003f493192fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:45.669357 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973040ef-f31b-44e4-bdfb-003f493192fc admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:45.674322 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2635b-d077-4aa9-9718-c3a68d879f64 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fd398c547554629927d2a80b3a8dee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:45.674955 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2635b-d077-4aa9-9718-c3a68d879f64 service glance] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:45.675229 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2635b-d077-4aa9-9718-c3a68d879f64 service glance] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:45.675476 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2635b-d077-4aa9-9718-c3a68d879f64 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:45.684375 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b2635b-d077-4aa9-9718-c3a68d879f64 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:45.685922 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b2635b-d077-4aa9-9718-c3a68d879f64 service glance] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:45.694791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1051/2125] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:45 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 09:48:50.729304 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a408b81-95b4-4be6-bbd3-a00334cee100 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:50.731106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-7a408b81-95b4-4be6-bbd3-a00334cee100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:50.804782 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a408b81-95b4-4be6-bbd3-a00334cee100 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:50.810060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0356e-73c4-4520-9cc3-881613c1f53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:50.810964 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0356e-73c4-4520-9cc3-881613c1f53f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:50.811238 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0356e-73c4-4520-9cc3-881613c1f53f admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:50.811515 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0356e-73c4-4520-9cc3-881613c1f53f admin admin] PATH_INFO: `/v3/users/6185da34cccc427ebc9f5ebc7d73ce13` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:50.812892 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c0356e-73c4-4520-9cc3-881613c1f53f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6185da34cccc427ebc9f5ebc7d73ce13)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:50.815121 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c0356e-73c4-4520-9cc3-881613c1f53f admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:50.864108 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-97c0356e-73c4-4520-9cc3-881613c1f53f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6185da34cccc427ebc9f5ebc7d73ce13'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:50.871203 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-97c0356e-73c4-4520-9cc3-881613c1f53f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6185da34cccc427ebc9f5ebc7d73ce13'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:50.871900 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-97c0356e-73c4-4520-9cc3-881613c1f53f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6185da34cccc427ebc9f5ebc7d73ce13'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:50.879785 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-97c0356e-73c4-4520-9cc3-881613c1f53f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6185da34cccc427ebc9f5ebc7d73ce13'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:50.880405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-97c0356e-73c4-4520-9cc3-881613c1f53f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6185da34cccc427ebc9f5ebc7d73ce13'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:50.897676 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1075/2126] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:50 2022] DELETE /identity/v3/users/6185da34cccc427ebc9f5ebc7d73ce13 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:50.902491 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf279c77-8cd6-4835-b67d-5e8ce43fc6d7 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:50.903081 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf279c77-8cd6-4835-b67d-5e8ce43fc6d7 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:50.903081 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf279c77-8cd6-4835-b67d-5e8ce43fc6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:50.922506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-bf279c77-8cd6-4835-b67d-5e8ce43fc6d7 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:50.923817 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-bf279c77-8cd6-4835-b67d-5e8ce43fc6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:50.999370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1052/2127] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:51.009090 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0356e-73c4-4520-9cc3-881613c1f53f admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:51.010515 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-97c0356e-73c4-4520-9cc3-881613c1f53f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:51.068371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0356e-73c4-4520-9cc3-881613c1f53f admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:51.072800 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1d734-8ca6-48eb-b235-0d16fdf9038e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:51.073454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1d734-8ca6-48eb-b235-0d16fdf9038e service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:51.073730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1d734-8ca6-48eb-b235-0d16fdf9038e service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:51.073998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1d734-8ca6-48eb-b235-0d16fdf9038e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:51.083559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1d734-8ca6-48eb-b235-0d16fdf9038e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:51.085404 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1d734-8ca6-48eb-b235-0d16fdf9038e service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:51.097916 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1076/2128] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:48:51.375771 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf279c77-8cd6-4835-b67d-5e8ce43fc6d7 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:51.377291 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-bf279c77-8cd6-4835-b67d-5e8ce43fc6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:51.390308 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf279c77-8cd6-4835-b67d-5e8ce43fc6d7 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:51.402599 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031dfc2-50f0-4de1-bf46-a1e0c5d7238d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:51.403743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031dfc2-50f0-4de1-bf46-a1e0c5d7238d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:51.403743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031dfc2-50f0-4de1-bf46-a1e0c5d7238d admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:51.404525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031dfc2-50f0-4de1-bf46-a1e0c5d7238d admin admin] PATH_INFO: `/v3/projects/2e1f17b52cb74bb486f33e49e0102aa3` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:51.406259 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031dfc2-50f0-4de1-bf46-a1e0c5d7238d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e1f17b52cb74bb486f33e49e0102aa3)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:51.422230 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031dfc2-50f0-4de1-bf46-a1e0c5d7238d admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:51.478156 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-5031dfc2-50f0-4de1-bf46-a1e0c5d7238d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e1f17b52cb74bb486f33e49e0102aa3', 'request_id': 'req-5031dfc2-50f0-4de1-bf46-a1e0c5d7238d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:51.484307 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-5031dfc2-50f0-4de1-bf46-a1e0c5d7238d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e1f17b52cb74bb486f33e49e0102aa3', 'request_id': 'req-5031dfc2-50f0-4de1-bf46-a1e0c5d7238d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:51.490291 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-5031dfc2-50f0-4de1-bf46-a1e0c5d7238d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e1f17b52cb74bb486f33e49e0102aa3', 'request_id': 'req-5031dfc2-50f0-4de1-bf46-a1e0c5d7238d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:51.503493 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-5031dfc2-50f0-4de1-bf46-a1e0c5d7238d admin admin] The token cache is being invalidate because project 2e1f17b52cb74bb486f33e49e0102aa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:51.503892 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-5031dfc2-50f0-4de1-bf46-a1e0c5d7238d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:51.504613 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-5031dfc2-50f0-4de1-bf46-a1e0c5d7238d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:51.510796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1053/2129] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:51 2022] DELETE /identity/v3/projects/2e1f17b52cb74bb486f33e49e0102aa3 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:51.515650 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1d734-8ca6-48eb-b235-0d16fdf9038e service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:51.528453 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1d734-8ca6-48eb-b235-0d16fdf9038e service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:51.587561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ad588-3a43-4811-93db-bcdbe13cc3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:51.588788 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ad588-3a43-4811-93db-bcdbe13cc3fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:51.589072 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ad588-3a43-4811-93db-bcdbe13cc3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:51.589390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ad588-3a43-4811-93db-bcdbe13cc3fb admin admin] PATH_INFO: `/v3/users/bb07d4e904a64a0f98758b42c31e8ece` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:51.590630 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ad588-3a43-4811-93db-bcdbe13cc3fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb07d4e904a64a0f98758b42c31e8ece)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:51.592597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ad588-3a43-4811-93db-bcdbe13cc3fb admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:51.640191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-a70ad588-3a43-4811-93db-bcdbe13cc3fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb07d4e904a64a0f98758b42c31e8ece'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:51.647010 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-a70ad588-3a43-4811-93db-bcdbe13cc3fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb07d4e904a64a0f98758b42c31e8ece'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:51.647719 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-a70ad588-3a43-4811-93db-bcdbe13cc3fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb07d4e904a64a0f98758b42c31e8ece'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:51.655149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-a70ad588-3a43-4811-93db-bcdbe13cc3fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb07d4e904a64a0f98758b42c31e8ece'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:51.655743 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-a70ad588-3a43-4811-93db-bcdbe13cc3fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb07d4e904a64a0f98758b42c31e8ece'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:51.671902 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1077/2130] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:51 2022] DELETE /identity/v3/users/bb07d4e904a64a0f98758b42c31e8ece => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:51.941628 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031dfc2-50f0-4de1-bf46-a1e0c5d7238d admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:51.943154 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-5031dfc2-50f0-4de1-bf46-a1e0c5d7238d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:51.957953 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031dfc2-50f0-4de1-bf46-a1e0c5d7238d admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:52.016686 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd341f7c-b749-4595-972b-2f98e509467f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:52.017624 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd341f7c-b749-4595-972b-2f98e509467f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:52.017877 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd341f7c-b749-4595-972b-2f98e509467f admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:52.018148 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd341f7c-b749-4595-972b-2f98e509467f admin admin] PATH_INFO: `/v3/projects/8b628c04ec2e424e90fce6dc7fcb97ec` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:52.019362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd341f7c-b749-4595-972b-2f98e509467f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b628c04ec2e424e90fce6dc7fcb97ec)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:52.021432 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd341f7c-b749-4595-972b-2f98e509467f admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:52.068507 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bd341f7c-b749-4595-972b-2f98e509467f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b628c04ec2e424e90fce6dc7fcb97ec', 'request_id': 'req-bd341f7c-b749-4595-972b-2f98e509467f'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:52.074566 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bd341f7c-b749-4595-972b-2f98e509467f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b628c04ec2e424e90fce6dc7fcb97ec', 'request_id': 'req-bd341f7c-b749-4595-972b-2f98e509467f'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:52.079689 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bd341f7c-b749-4595-972b-2f98e509467f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b628c04ec2e424e90fce6dc7fcb97ec', 'request_id': 'req-bd341f7c-b749-4595-972b-2f98e509467f'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:52.093045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bd341f7c-b749-4595-972b-2f98e509467f admin admin] The token cache is being invalidate because project 8b628c04ec2e424e90fce6dc7fcb97ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:52.093330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bd341f7c-b749-4595-972b-2f98e509467f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:52.093934 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bd341f7c-b749-4595-972b-2f98e509467f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:52.095479 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1054/2131] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:51 2022] DELETE /identity/v3/projects/8b628c04ec2e424e90fce6dc7fcb97ec => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:52.100419 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062d89f-9972-47d6-afda-51474504a79b None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:52.100945 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062d89f-9972-47d6-afda-51474504a79b None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:52.101583 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062d89f-9972-47d6-afda-51474504a79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:52.122328 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-9062d89f-9972-47d6-afda-51474504a79b None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:52.123545 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-9062d89f-9972-47d6-afda-51474504a79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:52.181201 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1078/2132] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:52.190141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff8ca3-1677-40c3-9369-821f07c8e960 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:52.190394 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff8ca3-1677-40c3-9369-821f07c8e960 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:52.190681 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff8ca3-1677-40c3-9369-821f07c8e960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:52.210096 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-e6ff8ca3-1677-40c3-9369-821f07c8e960 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:52.211315 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e6ff8ca3-1677-40c3-9369-821f07c8e960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:52.219387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1055/2133] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:52.223605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062d89f-9972-47d6-afda-51474504a79b None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:52.233362 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062d89f-9972-47d6-afda-51474504a79b None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:52.238694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d80de7-9989-4992-9277-1a376a57b39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:52.239475 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d80de7-9989-4992-9277-1a376a57b39d admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:52.239626 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d80de7-9989-4992-9277-1a376a57b39d admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:52.239890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d80de7-9989-4992-9277-1a376a57b39d admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:52.244556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d80de7-9989-4992-9277-1a376a57b39d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:48:52.244899 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d80de7-9989-4992-9277-1a376a57b39d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:52.246547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d80de7-9989-4992-9277-1a376a57b39d admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:52.258235 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1079/2134] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:48:52.262690 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff8ca3-1677-40c3-9369-821f07c8e960 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:52.272164 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff8ca3-1677-40c3-9369-821f07c8e960 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:52.277791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141002a-1457-49d4-ac04-4c48df78f59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:52.278439 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141002a-1457-49d4-ac04-4c48df78f59b admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:52.278714 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141002a-1457-49d4-ac04-4c48df78f59b admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:52.278977 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141002a-1457-49d4-ac04-4c48df78f59b admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:52.279665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141002a-1457-49d4-ac04-4c48df78f59b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:52.281632 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141002a-1457-49d4-ac04-4c48df78f59b admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:52.310525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1056/2135] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:48:52 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:52.314888 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d80de7-9989-4992-9277-1a376a57b39d admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:52.325784 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d80de7-9989-4992-9277-1a376a57b39d admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:52.383148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e721f-b6ed-4a80-86eb-384365d29791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:52.383808 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e721f-b6ed-4a80-86eb-384365d29791 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:52.384096 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e721f-b6ed-4a80-86eb-384365d29791 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:52.384374 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e721f-b6ed-4a80-86eb-384365d29791 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:52.385101 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e721f-b6ed-4a80-86eb-384365d29791 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:52.387186 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e721f-b6ed-4a80-86eb-384365d29791 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:52.411258 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1080/2136] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:48:52 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:48:52.412630 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141002a-1457-49d4-ac04-4c48df78f59b admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:52.422725 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141002a-1457-49d4-ac04-4c48df78f59b admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:52.427616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18dc13-e789-4d03-b895-72ac66234388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:52.428293 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18dc13-e789-4d03-b895-72ac66234388 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:52.428571 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18dc13-e789-4d03-b895-72ac66234388 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:52.428875 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18dc13-e789-4d03-b895-72ac66234388 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:52.429622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18dc13-e789-4d03-b895-72ac66234388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:52.431340 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18dc13-e789-4d03-b895-72ac66234388 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:52.441947 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1057/2137] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:52.446048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e721f-b6ed-4a80-86eb-384365d29791 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:52.456193 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e721f-b6ed-4a80-86eb-384365d29791 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:52.461063 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa8183-583a-4cd1-9628-1692dcdbe98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:52.461707 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa8183-583a-4cd1-9628-1692dcdbe98b admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:52.461983 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa8183-583a-4cd1-9628-1692dcdbe98b admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:52.462658 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa8183-583a-4cd1-9628-1692dcdbe98b admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:52.462962 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa8183-583a-4cd1-9628-1692dcdbe98b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:52.464557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa8183-583a-4cd1-9628-1692dcdbe98b admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:52.474983 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1081/2138] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:52.479242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18dc13-e789-4d03-b895-72ac66234388 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:52.489812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18dc13-e789-4d03-b895-72ac66234388 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:52.494860 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b0b39-8e96-420b-be20-06ec3eb95f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:52.495767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b0b39-8e96-420b-be20-06ec3eb95f2a admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:52.496041 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b0b39-8e96-420b-be20-06ec3eb95f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:52.496304 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b0b39-8e96-420b-be20-06ec3eb95f2a admin admin] PATH_INFO: `/v3/projects/b64a8fd2a069461d912036ce464a02f7/users/0b39d1be479f4493a46eb1a0c628aab3/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:52.511373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b0b39-8e96-420b-be20-06ec3eb95f2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b64a8fd2a069461d912036ce464a02f7, user_id=0b39d1be479f4493a46eb1a0c628aab3, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:52.514131 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b0b39-8e96-420b-be20-06ec3eb95f2a admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:52.528767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1058/2139] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:52 2022] PUT /identity/v3/projects/b64a8fd2a069461d912036ce464a02f7/users/0b39d1be479f4493a46eb1a0c628aab3/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:52.539414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a90631-a91d-46a8-8e6d-07b26719935a None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:52.539742 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a90631-a91d-46a8-8e6d-07b26719935a None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:52.540084 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a90631-a91d-46a8-8e6d-07b26719935a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:52.568494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-c4a90631-a91d-46a8-8e6d-07b26719935a None None] MFA Rules not processed for user `0b39d1be479f4493a46eb1a0c628aab3`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:52.569791 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c4a90631-a91d-46a8-8e6d-07b26719935a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:52.636409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1082/2140] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:52.643338 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b0b39-8e96-420b-be20-06ec3eb95f2a admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:52.710616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b0b39-8e96-420b-be20-06ec3eb95f2a admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:52.715790 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd3130-85b6-4dc6-b1c5-895d4d522cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:52.716444 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd3130-85b6-4dc6-b1c5-895d4d522cd9 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:52.716768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd3130-85b6-4dc6-b1c5-895d4d522cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:52.717025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd3130-85b6-4dc6-b1c5-895d4d522cd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:52.717716 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd3130-85b6-4dc6-b1c5-895d4d522cd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:52.719643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd3130-85b6-4dc6-b1c5-895d4d522cd9 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:52.747038 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1059/2141] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:48:52 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:48:52.751214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a90631-a91d-46a8-8e6d-07b26719935a None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:52.762014 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a90631-a91d-46a8-8e6d-07b26719935a None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:52.818062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ea392-ed02-4623-b28b-c869c5c01e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:52.818722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ea392-ed02-4623-b28b-c869c5c01e0b admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:52.819004 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ea392-ed02-4623-b28b-c869c5c01e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:52.819276 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ea392-ed02-4623-b28b-c869c5c01e0b admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:52.819966 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ea392-ed02-4623-b28b-c869c5c01e0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:52.821922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ea392-ed02-4623-b28b-c869c5c01e0b admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:52.844092 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1083/2142] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:48:52 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:48:52.848285 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd3130-85b6-4dc6-b1c5-895d4d522cd9 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:52.858659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd3130-85b6-4dc6-b1c5-895d4d522cd9 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:52.864041 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00fa79-630f-4c45-a900-ec6d84815334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:52.864718 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00fa79-630f-4c45-a900-ec6d84815334 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:52.865010 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00fa79-630f-4c45-a900-ec6d84815334 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:52.865276 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00fa79-630f-4c45-a900-ec6d84815334 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:52.866045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00fa79-630f-4c45-a900-ec6d84815334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:52.867637 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00fa79-630f-4c45-a900-ec6d84815334 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:52.878174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1060/2143] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:52.882193 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ea392-ed02-4623-b28b-c869c5c01e0b admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:52.892018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ea392-ed02-4623-b28b-c869c5c01e0b admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:52.897063 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33827491-aded-4e2c-ade6-3614916e4db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:52.897903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33827491-aded-4e2c-ade6-3614916e4db4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:52.898209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33827491-aded-4e2c-ade6-3614916e4db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:52.898476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33827491-aded-4e2c-ade6-3614916e4db4 admin admin] PATH_INFO: `/v3/projects/a68cb0ffd3624a21a193bc3f0b49bc92/users/23c0a359063c4128b5c005dbdeff5508/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:52.913901 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33827491-aded-4e2c-ade6-3614916e4db4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68cb0ffd3624a21a193bc3f0b49bc92, user_id=23c0a359063c4128b5c005dbdeff5508, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:52.916542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33827491-aded-4e2c-ade6-3614916e4db4 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:52.932117 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1084/2144] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:52 2022] PUT /identity/v3/projects/a68cb0ffd3624a21a193bc3f0b49bc92/users/23c0a359063c4128b5c005dbdeff5508/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:52.936535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00fa79-630f-4c45-a900-ec6d84815334 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:52.946527 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00fa79-630f-4c45-a900-ec6d84815334 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:53.010571 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346ae00-170d-481e-b0da-85291d2b9ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:53.011402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346ae00-170d-481e-b0da-85291d2b9ab1 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:53.011765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346ae00-170d-481e-b0da-85291d2b9ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:53.012052 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346ae00-170d-481e-b0da-85291d2b9ab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:53.012877 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346ae00-170d-481e-b0da-85291d2b9ab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:53.015247 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346ae00-170d-481e-b0da-85291d2b9ab1 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:53.026217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1061/2145] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:53.030723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33827491-aded-4e2c-ade6-3614916e4db4 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:53.041679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33827491-aded-4e2c-ade6-3614916e4db4 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:53.047824 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71289a7b-cc8d-41d7-a41d-3ed03cefb8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:53.048795 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71289a7b-cc8d-41d7-a41d-3ed03cefb8c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:53.049104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71289a7b-cc8d-41d7-a41d-3ed03cefb8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:53.049374 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71289a7b-cc8d-41d7-a41d-3ed03cefb8c1 admin admin] PATH_INFO: `/v3/projects/a68cb0ffd3624a21a193bc3f0b49bc92/users/23c0a359063c4128b5c005dbdeff5508/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:53.051648 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71289a7b-cc8d-41d7-a41d-3ed03cefb8c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68cb0ffd3624a21a193bc3f0b49bc92, user_id=23c0a359063c4128b5c005dbdeff5508, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:53.054395 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71289a7b-cc8d-41d7-a41d-3ed03cefb8c1 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:53.065411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1085/2146] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:53 2022] PUT /identity/v3/projects/a68cb0ffd3624a21a193bc3f0b49bc92/users/23c0a359063c4128b5c005dbdeff5508/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:53.082483 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea20e0f-7daf-44c9-9358-00f7de3be47d None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:53.082766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea20e0f-7daf-44c9-9358-00f7de3be47d None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:53.083027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea20e0f-7daf-44c9-9358-00f7de3be47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:53.111952 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-2ea20e0f-7daf-44c9-9358-00f7de3be47d None None] MFA Rules not processed for user `23c0a359063c4128b5c005dbdeff5508`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:53.113201 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-2ea20e0f-7daf-44c9-9358-00f7de3be47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:53.187081 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1062/2147] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:48:53.198157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71289a7b-cc8d-41d7-a41d-3ed03cefb8c1 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:53.201687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-71289a7b-cc8d-41d7-a41d-3ed03cefb8c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:53.264917 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71289a7b-cc8d-41d7-a41d-3ed03cefb8c1 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:53.271340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f16c7d-044f-40bb-8099-b3abd294eb8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:53.272008 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f16c7d-044f-40bb-8099-b3abd294eb8a service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:53.272917 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f16c7d-044f-40bb-8099-b3abd294eb8a service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:53.273195 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f16c7d-044f-40bb-8099-b3abd294eb8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:53.289564 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f16c7d-044f-40bb-8099-b3abd294eb8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:53.290989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f16c7d-044f-40bb-8099-b3abd294eb8a service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:53.300718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1086/2148] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:48:53.858816 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea20e0f-7daf-44c9-9358-00f7de3be47d None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:53.925379 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea20e0f-7daf-44c9-9358-00f7de3be47d None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:53.930566 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8626e-d26e-4081-85dc-3f0cced5e3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:53.931428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8626e-d26e-4081-85dc-3f0cced5e3cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:53.931701 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8626e-d26e-4081-85dc-3f0cced5e3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:53.931962 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8626e-d26e-4081-85dc-3f0cced5e3cd admin admin] PATH_INFO: `/v3/users/0b39d1be479f4493a46eb1a0c628aab3` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:53.933212 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca8626e-d26e-4081-85dc-3f0cced5e3cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b39d1be479f4493a46eb1a0c628aab3)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:53.935132 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca8626e-d26e-4081-85dc-3f0cced5e3cd admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:53.982836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-dca8626e-d26e-4081-85dc-3f0cced5e3cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b39d1be479f4493a46eb1a0c628aab3'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:53.990362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-dca8626e-d26e-4081-85dc-3f0cced5e3cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b39d1be479f4493a46eb1a0c628aab3'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:53.998993 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-dca8626e-d26e-4081-85dc-3f0cced5e3cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b39d1be479f4493a46eb1a0c628aab3'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:53.999627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-dca8626e-d26e-4081-85dc-3f0cced5e3cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b39d1be479f4493a46eb1a0c628aab3'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:54.000275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-dca8626e-d26e-4081-85dc-3f0cced5e3cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b39d1be479f4493a46eb1a0c628aab3'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:54.017308 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1063/2149] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:53 2022] DELETE /identity/v3/users/0b39d1be479f4493a46eb1a0c628aab3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:54.022185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773fbbc-4815-42ed-a7e3-08867063246e None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:54.022469 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773fbbc-4815-42ed-a7e3-08867063246e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:54.022734 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773fbbc-4815-42ed-a7e3-08867063246e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:54.040565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-8773fbbc-4815-42ed-a7e3-08867063246e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:54.041797 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-8773fbbc-4815-42ed-a7e3-08867063246e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:54.106922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1087/2150] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:54.118847 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8626e-d26e-4081-85dc-3f0cced5e3cd admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:54.120698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-dca8626e-d26e-4081-85dc-3f0cced5e3cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:54.181324 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8626e-d26e-4081-85dc-3f0cced5e3cd admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:54.186080 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fffc01-03f0-4815-850c-01827f027fa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:54.186718 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fffc01-03f0-4815-850c-01827f027fa1 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:54.186993 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fffc01-03f0-4815-850c-01827f027fa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:54.187612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fffc01-03f0-4815-850c-01827f027fa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:54.197195 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fffc01-03f0-4815-850c-01827f027fa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:54.198618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fffc01-03f0-4815-850c-01827f027fa1 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:54.208060 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1064/2151] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:48:54.496075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773fbbc-4815-42ed-a7e3-08867063246e None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:54.497662 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-8773fbbc-4815-42ed-a7e3-08867063246e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:54.510052 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773fbbc-4815-42ed-a7e3-08867063246e None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:54.515503 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab417f0-ab3a-4bfa-a463-5ebe9bafb9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:54.516377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab417f0-ab3a-4bfa-a463-5ebe9bafb9bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:54.516657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab417f0-ab3a-4bfa-a463-5ebe9bafb9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:54.516989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab417f0-ab3a-4bfa-a463-5ebe9bafb9bb admin admin] PATH_INFO: `/v3/projects/b64a8fd2a069461d912036ce464a02f7` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:54.518379 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab417f0-ab3a-4bfa-a463-5ebe9bafb9bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b64a8fd2a069461d912036ce464a02f7)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:54.520322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab417f0-ab3a-4bfa-a463-5ebe9bafb9bb admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:54.586187 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9ab417f0-ab3a-4bfa-a463-5ebe9bafb9bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b64a8fd2a069461d912036ce464a02f7', 'request_id': 'req-9ab417f0-ab3a-4bfa-a463-5ebe9bafb9bb'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:54.592640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9ab417f0-ab3a-4bfa-a463-5ebe9bafb9bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b64a8fd2a069461d912036ce464a02f7', 'request_id': 'req-9ab417f0-ab3a-4bfa-a463-5ebe9bafb9bb'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:54.606332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9ab417f0-ab3a-4bfa-a463-5ebe9bafb9bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b64a8fd2a069461d912036ce464a02f7', 'request_id': 'req-9ab417f0-ab3a-4bfa-a463-5ebe9bafb9bb'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:54.612159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9ab417f0-ab3a-4bfa-a463-5ebe9bafb9bb admin admin] The token cache is being invalidate because project b64a8fd2a069461d912036ce464a02f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:54.612561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9ab417f0-ab3a-4bfa-a463-5ebe9bafb9bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:54.613239 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9ab417f0-ab3a-4bfa-a463-5ebe9bafb9bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:54.614783 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1088/2152] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:54 2022] DELETE /identity/v3/projects/b64a8fd2a069461d912036ce464a02f7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:54.618696 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fffc01-03f0-4815-850c-01827f027fa1 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:54.628076 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fffc01-03f0-4815-850c-01827f027fa1 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:54.683583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62708f-ab59-4f2e-a8e1-d9c9075dc414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:54.684451 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62708f-ab59-4f2e-a8e1-d9c9075dc414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:54.684743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62708f-ab59-4f2e-a8e1-d9c9075dc414 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:54.685011 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62708f-ab59-4f2e-a8e1-d9c9075dc414 admin admin] PATH_INFO: `/v3/users/23c0a359063c4128b5c005dbdeff5508` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:54.686266 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62708f-ab59-4f2e-a8e1-d9c9075dc414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23c0a359063c4128b5c005dbdeff5508)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:54.688194 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62708f-ab59-4f2e-a8e1-d9c9075dc414 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:54.736572 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-7d62708f-ab59-4f2e-a8e1-d9c9075dc414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23c0a359063c4128b5c005dbdeff5508'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:54.743208 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-7d62708f-ab59-4f2e-a8e1-d9c9075dc414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23c0a359063c4128b5c005dbdeff5508'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:54.751322 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-7d62708f-ab59-4f2e-a8e1-d9c9075dc414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23c0a359063c4128b5c005dbdeff5508'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:54.752029 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-7d62708f-ab59-4f2e-a8e1-d9c9075dc414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23c0a359063c4128b5c005dbdeff5508'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:54.752627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-7d62708f-ab59-4f2e-a8e1-d9c9075dc414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23c0a359063c4128b5c005dbdeff5508'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:54.771036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1065/2153] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:48:54 2022] DELETE /identity/v3/users/23c0a359063c4128b5c005dbdeff5508 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:55.042314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab417f0-ab3a-4bfa-a463-5ebe9bafb9bb admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:55.043850 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-9ab417f0-ab3a-4bfa-a463-5ebe9bafb9bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:55.057498 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab417f0-ab3a-4bfa-a463-5ebe9bafb9bb admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:55.118560 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4374c8-5818-42ef-aa68-6e8ac7942b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:55.119344 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4374c8-5818-42ef-aa68-6e8ac7942b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:55.119615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4374c8-5818-42ef-aa68-6e8ac7942b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:55.119881 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4374c8-5818-42ef-aa68-6e8ac7942b20 admin admin] PATH_INFO: `/v3/projects/a68cb0ffd3624a21a193bc3f0b49bc92` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:55.121181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4374c8-5818-42ef-aa68-6e8ac7942b20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a68cb0ffd3624a21a193bc3f0b49bc92)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:55.123134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4374c8-5818-42ef-aa68-6e8ac7942b20 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:55.171582 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-1f4374c8-5818-42ef-aa68-6e8ac7942b20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a68cb0ffd3624a21a193bc3f0b49bc92', 'request_id': 'req-1f4374c8-5818-42ef-aa68-6e8ac7942b20'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:55.177719 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-1f4374c8-5818-42ef-aa68-6e8ac7942b20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a68cb0ffd3624a21a193bc3f0b49bc92', 'request_id': 'req-1f4374c8-5818-42ef-aa68-6e8ac7942b20'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:55.182975 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-1f4374c8-5818-42ef-aa68-6e8ac7942b20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a68cb0ffd3624a21a193bc3f0b49bc92', 'request_id': 'req-1f4374c8-5818-42ef-aa68-6e8ac7942b20'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:55.197249 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-1f4374c8-5818-42ef-aa68-6e8ac7942b20 admin admin] The token cache is being invalidate because project a68cb0ffd3624a21a193bc3f0b49bc92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:48:55.197531 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-1f4374c8-5818-42ef-aa68-6e8ac7942b20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:55.198160 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-1f4374c8-5818-42ef-aa68-6e8ac7942b20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:48:55.199725 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1089/2154] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:48:55 2022] DELETE /identity/v3/projects/a68cb0ffd3624a21a193bc3f0b49bc92 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:48:55.204814 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b33e79-94e1-4a59-8860-94dfa74d3477 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:55.205312 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b33e79-94e1-4a59-8860-94dfa74d3477 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:55.205312 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b33e79-94e1-4a59-8860-94dfa74d3477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:55.223934 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-e1b33e79-94e1-4a59-8860-94dfa74d3477 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:55.225183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e1b33e79-94e1-4a59-8860-94dfa74d3477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:55.285319 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1066/2155] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:55.294642 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc2ef6-54fc-40dd-8f57-8201b0390530 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:55.295154 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc2ef6-54fc-40dd-8f57-8201b0390530 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:55.295632 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc2ef6-54fc-40dd-8f57-8201b0390530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:55.318248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-9ffc2ef6-54fc-40dd-8f57-8201b0390530 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:55.319566 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-9ffc2ef6-54fc-40dd-8f57-8201b0390530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:55.329969 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1090/2156] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:48:55.334257 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b33e79-94e1-4a59-8860-94dfa74d3477 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:55.344709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b33e79-94e1-4a59-8860-94dfa74d3477 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:55.349855 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d16d0d-f3f8-4533-af3e-5ac386d0621b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:55.350512 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d16d0d-f3f8-4533-af3e-5ac386d0621b admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:55.350789 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d16d0d-f3f8-4533-af3e-5ac386d0621b admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:55.351059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d16d0d-f3f8-4533-af3e-5ac386d0621b admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:55.351787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d16d0d-f3f8-4533-af3e-5ac386d0621b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:48:55.352103 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d16d0d-f3f8-4533-af3e-5ac386d0621b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:55.353711 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d16d0d-f3f8-4533-af3e-5ac386d0621b admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:55.368389 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1067/2157] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:48:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:48:55.372982 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc2ef6-54fc-40dd-8f57-8201b0390530 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:55.382820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc2ef6-54fc-40dd-8f57-8201b0390530 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:55.387828 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758cae0-e7a3-4881-a0a9-cc3f71fa412a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:55.388491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758cae0-e7a3-4881-a0a9-cc3f71fa412a admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:55.388792 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758cae0-e7a3-4881-a0a9-cc3f71fa412a admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:55.389060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758cae0-e7a3-4881-a0a9-cc3f71fa412a admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:55.389756 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758cae0-e7a3-4881-a0a9-cc3f71fa412a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:55.391695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758cae0-e7a3-4881-a0a9-cc3f71fa412a admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:55.422777 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1091/2158] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:48:55 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:48:55.424113 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d16d0d-f3f8-4533-af3e-5ac386d0621b admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:55.434098 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d16d0d-f3f8-4533-af3e-5ac386d0621b admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:55.495396 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86abcfe9-0556-4832-ac4e-d224c35d0818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:55.496073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abcfe9-0556-4832-ac4e-d224c35d0818 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:55.496349 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abcfe9-0556-4832-ac4e-d224c35d0818 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:55.496614 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abcfe9-0556-4832-ac4e-d224c35d0818 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:55.497340 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86abcfe9-0556-4832-ac4e-d224c35d0818 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:55.499273 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86abcfe9-0556-4832-ac4e-d224c35d0818 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:55.523888 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1068/2159] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:48:55 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:48:55.529739 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758cae0-e7a3-4881-a0a9-cc3f71fa412a admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:55.540921 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758cae0-e7a3-4881-a0a9-cc3f71fa412a admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:55.546461 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c953a-d9ed-4bd6-8bb5-4917cb31703d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:55.547397 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c953a-d9ed-4bd6-8bb5-4917cb31703d admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:55.547682 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c953a-d9ed-4bd6-8bb5-4917cb31703d admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:55.547957 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c953a-d9ed-4bd6-8bb5-4917cb31703d admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:55.548718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c953a-d9ed-4bd6-8bb5-4917cb31703d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:55.550372 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c953a-d9ed-4bd6-8bb5-4917cb31703d admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:55.562757 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1092/2160] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:55.569629 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86abcfe9-0556-4832-ac4e-d224c35d0818 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:55.585633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86abcfe9-0556-4832-ac4e-d224c35d0818 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:55.590368 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f9e98-bfac-4c0e-a0bd-b863fd0ff747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:55.591039 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f9e98-bfac-4c0e-a0bd-b863fd0ff747 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:55.591314 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f9e98-bfac-4c0e-a0bd-b863fd0ff747 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:55.591582 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f9e98-bfac-4c0e-a0bd-b863fd0ff747 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:55.592312 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f9e98-bfac-4c0e-a0bd-b863fd0ff747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:55.594078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f9e98-bfac-4c0e-a0bd-b863fd0ff747 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:55.611359 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c953a-d9ed-4bd6-8bb5-4917cb31703d admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:55.620383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1069/2161] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:48:55.622758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c953a-d9ed-4bd6-8bb5-4917cb31703d admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:55.632915 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef7658-9f42-4c80-a6bb-ab190101bf10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:55.634556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef7658-9f42-4c80-a6bb-ab190101bf10 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:55.634556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef7658-9f42-4c80-a6bb-ab190101bf10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:55.634780 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef7658-9f42-4c80-a6bb-ab190101bf10 admin admin] PATH_INFO: `/v3/projects/9a382679a3a042ecb1fb011fe7c642f8/users/5c56cde8b9464350b83fb6824297b077/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:55.657416 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef7658-9f42-4c80-a6bb-ab190101bf10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a382679a3a042ecb1fb011fe7c642f8, user_id=5c56cde8b9464350b83fb6824297b077, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:55.660508 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef7658-9f42-4c80-a6bb-ab190101bf10 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:55.680700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1093/2162] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:48:55 2022] PUT /identity/v3/projects/9a382679a3a042ecb1fb011fe7c642f8/users/5c56cde8b9464350b83fb6824297b077/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:48:55.693897 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef856429-d879-42c4-b6d3-9c72ff301f38 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:55.694183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef856429-d879-42c4-b6d3-9c72ff301f38 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:55.694451 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef856429-d879-42c4-b6d3-9c72ff301f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:55.723072 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-ef856429-d879-42c4-b6d3-9c72ff301f38 None None] MFA Rules not processed for user `5c56cde8b9464350b83fb6824297b077`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:48:55.724368 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-ef856429-d879-42c4-b6d3-9c72ff301f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:55.797296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1070/2163] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:48:55 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:48:55.808712 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef7658-9f42-4c80-a6bb-ab190101bf10 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:48:55.810523 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e1ef7658-9f42-4c80-a6bb-ab190101bf10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:48:55.877609 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef7658-9f42-4c80-a6bb-ab190101bf10 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:48:55.881194 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0c7fc-9e18-4d2a-b006-4089bc89b349 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fd398c547554629927d2a80b3a8dee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:48:55.881865 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0c7fc-9e18-4d2a-b006-4089bc89b349 service glance] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:48:55.882139 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0c7fc-9e18-4d2a-b006-4089bc89b349 service glance] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:48:55.882395 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0c7fc-9e18-4d2a-b006-4089bc89b349 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:48:55.890444 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb0c7fc-9e18-4d2a-b006-4089bc89b349 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:48:55.891863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb0c7fc-9e18-4d2a-b006-4089bc89b349 service glance] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:48:55.900680 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1094/2164] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:48:55 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 09:49:00.077710 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef856429-d879-42c4-b6d3-9c72ff301f38 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:00.079267 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-ef856429-d879-42c4-b6d3-9c72ff301f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:00.155110 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef856429-d879-42c4-b6d3-9c72ff301f38 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:00.160342 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5714cf-f7ba-4c19-a4b3-13019ba20ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:00.161248 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5714cf-f7ba-4c19-a4b3-13019ba20ae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:00.161518 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5714cf-f7ba-4c19-a4b3-13019ba20ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:00.161784 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5714cf-f7ba-4c19-a4b3-13019ba20ae5 admin admin] PATH_INFO: `/v3/users/40cb55020f07488c9c344541d895ec1c` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:00.163176 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5714cf-f7ba-4c19-a4b3-13019ba20ae5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40cb55020f07488c9c344541d895ec1c)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:00.165256 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5714cf-f7ba-4c19-a4b3-13019ba20ae5 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:00.218012 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bd5714cf-f7ba-4c19-a4b3-13019ba20ae5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40cb55020f07488c9c344541d895ec1c'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:00.226359 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bd5714cf-f7ba-4c19-a4b3-13019ba20ae5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40cb55020f07488c9c344541d895ec1c'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:00.236090 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bd5714cf-f7ba-4c19-a4b3-13019ba20ae5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40cb55020f07488c9c344541d895ec1c'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:00.236797 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bd5714cf-f7ba-4c19-a4b3-13019ba20ae5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40cb55020f07488c9c344541d895ec1c'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:00.237495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bd5714cf-f7ba-4c19-a4b3-13019ba20ae5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40cb55020f07488c9c344541d895ec1c'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:00.255809 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1071/2165] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:49:00 2022] DELETE /identity/v3/users/40cb55020f07488c9c344541d895ec1c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:49:00.433622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0c7fc-9e18-4d2a-b006-4089bc89b349 service glance] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:00.446040 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0c7fc-9e18-4d2a-b006-4089bc89b349 service glance] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:00.502308 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd41b1-0846-4e6f-a843-0c077364a6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:00.503182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd41b1-0846-4e6f-a843-0c077364a6d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:00.503461 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd41b1-0846-4e6f-a843-0c077364a6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:00.503731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd41b1-0846-4e6f-a843-0c077364a6d2 admin admin] PATH_INFO: `/v3/projects/153cae04bb6b46d0a52c15411f0c4720` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:00.505032 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd41b1-0846-4e6f-a843-0c077364a6d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=153cae04bb6b46d0a52c15411f0c4720)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:00.506947 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd41b1-0846-4e6f-a843-0c077364a6d2 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:00.559558 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9bfd41b1-0846-4e6f-a843-0c077364a6d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '153cae04bb6b46d0a52c15411f0c4720', 'request_id': 'req-9bfd41b1-0846-4e6f-a843-0c077364a6d2'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:00.566289 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9bfd41b1-0846-4e6f-a843-0c077364a6d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '153cae04bb6b46d0a52c15411f0c4720', 'request_id': 'req-9bfd41b1-0846-4e6f-a843-0c077364a6d2'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:00.571963 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9bfd41b1-0846-4e6f-a843-0c077364a6d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '153cae04bb6b46d0a52c15411f0c4720', 'request_id': 'req-9bfd41b1-0846-4e6f-a843-0c077364a6d2'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:00.585848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9bfd41b1-0846-4e6f-a843-0c077364a6d2 admin admin] The token cache is being invalidate because project 153cae04bb6b46d0a52c15411f0c4720 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:49:00.586125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9bfd41b1-0846-4e6f-a843-0c077364a6d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:00.586744 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9bfd41b1-0846-4e6f-a843-0c077364a6d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:00.588350 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1095/2166] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:49:00 2022] DELETE /identity/v3/projects/153cae04bb6b46d0a52c15411f0c4720 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:49:00.593347 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de3b35-f22e-4d01-9a93-665b21ea3700 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:00.593632 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de3b35-f22e-4d01-9a93-665b21ea3700 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:00.593907 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de3b35-f22e-4d01-9a93-665b21ea3700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:00.612081 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-77de3b35-f22e-4d01-9a93-665b21ea3700 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:00.613551 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-77de3b35-f22e-4d01-9a93-665b21ea3700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:00.675974 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1072/2167] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:49:00.684467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f374859-e9b5-45b4-aade-d3d74b94a115 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:00.684804 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f374859-e9b5-45b4-aade-d3d74b94a115 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:00.685081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f374859-e9b5-45b4-aade-d3d74b94a115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:00.705693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-7f374859-e9b5-45b4-aade-d3d74b94a115 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:00.706949 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-7f374859-e9b5-45b4-aade-d3d74b94a115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:00.717007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1096/2168] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:49:00.721491 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de3b35-f22e-4d01-9a93-665b21ea3700 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:00.731671 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de3b35-f22e-4d01-9a93-665b21ea3700 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:00.737984 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d733c2-1551-46e8-8bc2-2e5aac21e49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:00.738692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d733c2-1551-46e8-8bc2-2e5aac21e49a admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:00.738955 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d733c2-1551-46e8-8bc2-2e5aac21e49a admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:00.739214 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d733c2-1551-46e8-8bc2-2e5aac21e49a admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:00.739924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d733c2-1551-46e8-8bc2-2e5aac21e49a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:49:00.740228 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d733c2-1551-46e8-8bc2-2e5aac21e49a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:00.741911 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d733c2-1551-46e8-8bc2-2e5aac21e49a admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:00.754006 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1073/2169] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:49:00.758798 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f374859-e9b5-45b4-aade-d3d74b94a115 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:00.768632 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f374859-e9b5-45b4-aade-d3d74b94a115 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:00.774673 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61d3c4-3b47-4382-9649-80c9215956c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:00.775352 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61d3c4-3b47-4382-9649-80c9215956c4 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:00.775632 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61d3c4-3b47-4382-9649-80c9215956c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:00.775896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61d3c4-3b47-4382-9649-80c9215956c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:00.776600 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61d3c4-3b47-4382-9649-80c9215956c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:00.778560 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61d3c4-3b47-4382-9649-80c9215956c4 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:00.808605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1097/2170] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:49:00 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:49:00.812864 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d733c2-1551-46e8-8bc2-2e5aac21e49a admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:00.823002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d733c2-1551-46e8-8bc2-2e5aac21e49a admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:00.880894 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a0662-2bf3-4fe4-895c-29e9d1521d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:00.881556 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a0662-2bf3-4fe4-895c-29e9d1521d85 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:00.881831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a0662-2bf3-4fe4-895c-29e9d1521d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:00.882094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a0662-2bf3-4fe4-895c-29e9d1521d85 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:00.882791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a0662-2bf3-4fe4-895c-29e9d1521d85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:00.884783 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a0662-2bf3-4fe4-895c-29e9d1521d85 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:00.909458 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1074/2171] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:49:00 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:49:00.913780 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61d3c4-3b47-4382-9649-80c9215956c4 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:00.924420 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61d3c4-3b47-4382-9649-80c9215956c4 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:00.929786 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9598da6-5e6a-4dce-b708-6deeaf9f72ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:00.930457 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9598da6-5e6a-4dce-b708-6deeaf9f72ee admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:00.931150 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9598da6-5e6a-4dce-b708-6deeaf9f72ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:00.931264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9598da6-5e6a-4dce-b708-6deeaf9f72ee admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:00.931997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9598da6-5e6a-4dce-b708-6deeaf9f72ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:00.933922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9598da6-5e6a-4dce-b708-6deeaf9f72ee admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:00.944416 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1098/2172] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:49:00.948579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a0662-2bf3-4fe4-895c-29e9d1521d85 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:00.959025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a0662-2bf3-4fe4-895c-29e9d1521d85 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:00.964337 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b11f40-9778-4db3-a346-a08b0f0c5f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:00.965051 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b11f40-9778-4db3-a346-a08b0f0c5f02 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:00.965353 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b11f40-9778-4db3-a346-a08b0f0c5f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:00.965626 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b11f40-9778-4db3-a346-a08b0f0c5f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:00.966357 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b11f40-9778-4db3-a346-a08b0f0c5f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:00.968050 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b11f40-9778-4db3-a346-a08b0f0c5f02 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:00.978436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1075/2173] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:49:00.982680 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9598da6-5e6a-4dce-b708-6deeaf9f72ee admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:00.993380 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9598da6-5e6a-4dce-b708-6deeaf9f72ee admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:00.999908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78c9bd-73b5-45a4-91e1-26ddb79517d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:01.000846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78c9bd-73b5-45a4-91e1-26ddb79517d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:01.001101 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78c9bd-73b5-45a4-91e1-26ddb79517d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:01.001365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78c9bd-73b5-45a4-91e1-26ddb79517d7 admin admin] PATH_INFO: `/v3/projects/e79a219bf03c403aa5d25f0f633f324c/users/cf1c6e68a1ea45ce9e390f07cf7cf505/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:01.017693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78c9bd-73b5-45a4-91e1-26ddb79517d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e79a219bf03c403aa5d25f0f633f324c, user_id=cf1c6e68a1ea45ce9e390f07cf7cf505, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:01.020609 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78c9bd-73b5-45a4-91e1-26ddb79517d7 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:01.042801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1099/2174] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:49:00 2022] PUT /identity/v3/projects/e79a219bf03c403aa5d25f0f633f324c/users/cf1c6e68a1ea45ce9e390f07cf7cf505/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:49:01.046176 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12532d7a-0619-469d-882f-ad476d2e7ab1 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:01.046450 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12532d7a-0619-469d-882f-ad476d2e7ab1 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:01.046722 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12532d7a-0619-469d-882f-ad476d2e7ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:01.076032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-12532d7a-0619-469d-882f-ad476d2e7ab1 None None] MFA Rules not processed for user `cf1c6e68a1ea45ce9e390f07cf7cf505`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:01.077262 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-12532d7a-0619-469d-882f-ad476d2e7ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:01.147604 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1076/2175] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:49:01.160337 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78c9bd-73b5-45a4-91e1-26ddb79517d7 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:01.162225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-fe78c9bd-73b5-45a4-91e1-26ddb79517d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:01.243802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78c9bd-73b5-45a4-91e1-26ddb79517d7 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:01.249438 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d4d0f-7ab6-4b41-9b55-ae7cd5fbbbd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:01.250095 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d4d0f-7ab6-4b41-9b55-ae7cd5fbbbd6 service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:01.250370 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d4d0f-7ab6-4b41-9b55-ae7cd5fbbbd6 service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:01.250669 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d4d0f-7ab6-4b41-9b55-ae7cd5fbbbd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:01.258416 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d4d0f-7ab6-4b41-9b55-ae7cd5fbbbd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:01.260130 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d4d0f-7ab6-4b41-9b55-ae7cd5fbbbd6 service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:01.269186 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1100/2176] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:49:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:49:29.766094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12532d7a-0619-469d-882f-ad476d2e7ab1 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:29.873199 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12532d7a-0619-469d-882f-ad476d2e7ab1 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:29.878997 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327f042-aaeb-433a-96b8-dc439dd773c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:29.879939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327f042-aaeb-433a-96b8-dc439dd773c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:29.880214 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327f042-aaeb-433a-96b8-dc439dd773c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:29.880481 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327f042-aaeb-433a-96b8-dc439dd773c4 admin admin] PATH_INFO: `/v3/users/cf1c6e68a1ea45ce9e390f07cf7cf505` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:29.881858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327f042-aaeb-433a-96b8-dc439dd773c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf1c6e68a1ea45ce9e390f07cf7cf505)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:29.883826 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327f042-aaeb-433a-96b8-dc439dd773c4 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:29.935499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-d327f042-aaeb-433a-96b8-dc439dd773c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf1c6e68a1ea45ce9e390f07cf7cf505'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:29.943206 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-d327f042-aaeb-433a-96b8-dc439dd773c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf1c6e68a1ea45ce9e390f07cf7cf505'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:29.951365 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-d327f042-aaeb-433a-96b8-dc439dd773c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf1c6e68a1ea45ce9e390f07cf7cf505'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:29.951978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-d327f042-aaeb-433a-96b8-dc439dd773c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf1c6e68a1ea45ce9e390f07cf7cf505'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:29.952684 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-d327f042-aaeb-433a-96b8-dc439dd773c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf1c6e68a1ea45ce9e390f07cf7cf505'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:29.970761 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1077/2177] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:49:29 2022] DELETE /identity/v3/users/cf1c6e68a1ea45ce9e390f07cf7cf505 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:49:29.975442 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d543bc-d689-4316-a29e-92e1980e25ec None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:29.976099 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d543bc-d689-4316-a29e-92e1980e25ec None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:29.976099 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d543bc-d689-4316-a29e-92e1980e25ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:30.029500 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-45d543bc-d689-4316-a29e-92e1980e25ec None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:30.030692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-45d543bc-d689-4316-a29e-92e1980e25ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:30.092068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1101/2178] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:49:30.099601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327f042-aaeb-433a-96b8-dc439dd773c4 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:30.101105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-d327f042-aaeb-433a-96b8-dc439dd773c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:30.197711 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327f042-aaeb-433a-96b8-dc439dd773c4 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:30.201511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4ced0-1d48-4b2d-aef6-00f59a38d59f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:30.202174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4ced0-1d48-4b2d-aef6-00f59a38d59f service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:30.202446 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4ced0-1d48-4b2d-aef6-00f59a38d59f service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:30.202715 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4ced0-1d48-4b2d-aef6-00f59a38d59f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:30.212441 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4ced0-1d48-4b2d-aef6-00f59a38d59f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:30.213932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4ced0-1d48-4b2d-aef6-00f59a38d59f service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:30.223716 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1078/2179] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:49:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 09:49:30.502720 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d543bc-d689-4316-a29e-92e1980e25ec None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:30.504192 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-45d543bc-d689-4316-a29e-92e1980e25ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:30.518130 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d543bc-d689-4316-a29e-92e1980e25ec None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:30.523625 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad38cb6-81f1-49e5-b194-6e2e6bc8a445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:30.524494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad38cb6-81f1-49e5-b194-6e2e6bc8a445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:30.524827 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad38cb6-81f1-49e5-b194-6e2e6bc8a445 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:30.525123 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad38cb6-81f1-49e5-b194-6e2e6bc8a445 admin admin] PATH_INFO: `/v3/projects/e79a219bf03c403aa5d25f0f633f324c` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:30.526468 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad38cb6-81f1-49e5-b194-6e2e6bc8a445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e79a219bf03c403aa5d25f0f633f324c)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:30.528430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad38cb6-81f1-49e5-b194-6e2e6bc8a445 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:30.580882 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bad38cb6-81f1-49e5-b194-6e2e6bc8a445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e79a219bf03c403aa5d25f0f633f324c', 'request_id': 'req-bad38cb6-81f1-49e5-b194-6e2e6bc8a445'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:30.587229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bad38cb6-81f1-49e5-b194-6e2e6bc8a445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e79a219bf03c403aa5d25f0f633f324c', 'request_id': 'req-bad38cb6-81f1-49e5-b194-6e2e6bc8a445'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:30.594605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bad38cb6-81f1-49e5-b194-6e2e6bc8a445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e79a219bf03c403aa5d25f0f633f324c', 'request_id': 'req-bad38cb6-81f1-49e5-b194-6e2e6bc8a445'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:30.610756 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bad38cb6-81f1-49e5-b194-6e2e6bc8a445 admin admin] The token cache is being invalidate because project e79a219bf03c403aa5d25f0f633f324c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:49:30.611044 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bad38cb6-81f1-49e5-b194-6e2e6bc8a445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:30.611729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bad38cb6-81f1-49e5-b194-6e2e6bc8a445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:30.613407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1102/2180] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:49:30 2022] DELETE /identity/v3/projects/e79a219bf03c403aa5d25f0f633f324c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:49:30.619945 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f34a7-7bed-40ef-9473-5ecff99420aa None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:30.620461 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f34a7-7bed-40ef-9473-5ecff99420aa None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:30.620948 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f34a7-7bed-40ef-9473-5ecff99420aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:30.666720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-a02f34a7-7bed-40ef-9473-5ecff99420aa None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:30.668640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a02f34a7-7bed-40ef-9473-5ecff99420aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:30.737422 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1079/2181] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:49:30.745397 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba5492-1fe7-4db9-81b4-85718d2725f1 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:30.745681 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba5492-1fe7-4db9-81b4-85718d2725f1 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:30.745950 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba5492-1fe7-4db9-81b4-85718d2725f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:30.772140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-95ba5492-1fe7-4db9-81b4-85718d2725f1 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:30.774217 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-95ba5492-1fe7-4db9-81b4-85718d2725f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:30.787211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1103/2182] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:49:30.792434 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f34a7-7bed-40ef-9473-5ecff99420aa None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:30.807384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f34a7-7bed-40ef-9473-5ecff99420aa None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:30.814822 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed767f9-9164-4e34-97eb-e99c4f28b5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:30.815468 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed767f9-9164-4e34-97eb-e99c4f28b5f5 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:30.815739 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed767f9-9164-4e34-97eb-e99c4f28b5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:30.816000 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed767f9-9164-4e34-97eb-e99c4f28b5f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:30.816754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed767f9-9164-4e34-97eb-e99c4f28b5f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:49:30.817051 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed767f9-9164-4e34-97eb-e99c4f28b5f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:30.818662 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed767f9-9164-4e34-97eb-e99c4f28b5f5 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:30.831314 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1080/2183] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:49:30.835768 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba5492-1fe7-4db9-81b4-85718d2725f1 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:30.845323 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba5492-1fe7-4db9-81b4-85718d2725f1 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:30.850867 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf7ef1-380b-4ece-a7b3-b153bb807110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:30.851562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf7ef1-380b-4ece-a7b3-b153bb807110 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:30.851835 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf7ef1-380b-4ece-a7b3-b153bb807110 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:30.852125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf7ef1-380b-4ece-a7b3-b153bb807110 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:30.852862 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf7ef1-380b-4ece-a7b3-b153bb807110 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:30.854760 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf7ef1-380b-4ece-a7b3-b153bb807110 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:30.883283 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1104/2184] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:49:30 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:49:30.887585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed767f9-9164-4e34-97eb-e99c4f28b5f5 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:30.898257 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed767f9-9164-4e34-97eb-e99c4f28b5f5 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:30.962511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb23c5f-083a-4269-8426-df0454e6c99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:30.963762 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb23c5f-083a-4269-8426-df0454e6c99d admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:30.964253 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb23c5f-083a-4269-8426-df0454e6c99d admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:30.964744 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb23c5f-083a-4269-8426-df0454e6c99d admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:30.965859 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb23c5f-083a-4269-8426-df0454e6c99d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:30.969145 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb23c5f-083a-4269-8426-df0454e6c99d admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:30.996494 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1081/2185] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:49:30 2022] POST /identity/v3/users => generated 284 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:49:31.001227 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf7ef1-380b-4ece-a7b3-b153bb807110 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:31.012040 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf7ef1-380b-4ece-a7b3-b153bb807110 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:31.017178 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a16f3-ff44-4a75-b42b-3c3b0927e3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:31.017841 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a16f3-ff44-4a75-b42b-3c3b0927e3e6 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:31.018115 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a16f3-ff44-4a75-b42b-3c3b0927e3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:31.018382 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a16f3-ff44-4a75-b42b-3c3b0927e3e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:31.019146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a16f3-ff44-4a75-b42b-3c3b0927e3e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:31.020784 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a16f3-ff44-4a75-b42b-3c3b0927e3e6 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:31.031519 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1105/2186] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:49:31.035806 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb23c5f-083a-4269-8426-df0454e6c99d admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:31.045336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb23c5f-083a-4269-8426-df0454e6c99d admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:31.050467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3734a-ee82-4e4e-93e9-c12557cd8192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:31.051196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3734a-ee82-4e4e-93e9-c12557cd8192 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:31.051540 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3734a-ee82-4e4e-93e9-c12557cd8192 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:31.051869 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3734a-ee82-4e4e-93e9-c12557cd8192 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:31.052700 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3734a-ee82-4e4e-93e9-c12557cd8192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:31.054382 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3734a-ee82-4e4e-93e9-c12557cd8192 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:31.064524 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1082/2187] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:49:31.070430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a16f3-ff44-4a75-b42b-3c3b0927e3e6 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:31.085737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a16f3-ff44-4a75-b42b-3c3b0927e3e6 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:31.092875 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ecfff4-c7ea-4663-ad76-103e988f9070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:31.094084 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ecfff4-c7ea-4663-ad76-103e988f9070 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:31.094423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ecfff4-c7ea-4663-ad76-103e988f9070 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:31.094920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ecfff4-c7ea-4663-ad76-103e988f9070 admin admin] PATH_INFO: `/v3/projects/91744b9cc5b14da1949db5dc161b433c/users/085db625954b4869a7f81ce2eb0416f0/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:31.110093 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ecfff4-c7ea-4663-ad76-103e988f9070 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91744b9cc5b14da1949db5dc161b433c, user_id=085db625954b4869a7f81ce2eb0416f0, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:31.112784 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ecfff4-c7ea-4663-ad76-103e988f9070 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:31.127827 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1106/2188] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:49:31 2022] PUT /identity/v3/projects/91744b9cc5b14da1949db5dc161b433c/users/085db625954b4869a7f81ce2eb0416f0/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:49:31.138283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d8524-429a-4405-9eee-d7786fbe2974 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:31.138586 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d8524-429a-4405-9eee-d7786fbe2974 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:31.138828 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d8524-429a-4405-9eee-d7786fbe2974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:31.167431 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-ab2d8524-429a-4405-9eee-d7786fbe2974 None None] MFA Rules not processed for user `085db625954b4869a7f81ce2eb0416f0`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:31.168620 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-ab2d8524-429a-4405-9eee-d7786fbe2974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:31.230146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc59b03-805a-421c-a23e-8c24e3c11d1c None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:31.230509 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc59b03-805a-421c-a23e-8c24e3c11d1c None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:31.230698 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc59b03-805a-421c-a23e-8c24e3c11d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:31.234298 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1083/2189] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:31 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:49:31.245098 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d8524-429a-4405-9eee-d7786fbe2974 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:31.246968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-ab2d8524-429a-4405-9eee-d7786fbe2974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:31.249632 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-edc59b03-805a-421c-a23e-8c24e3c11d1c None None] MFA Rules not processed for user `1c091a71e2ad46ceafe25be874c016a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:31.250859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-edc59b03-805a-421c-a23e-8c24e3c11d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:31.306917 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d8524-429a-4405-9eee-d7786fbe2974 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:31.310024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1107/2190] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:49:31.310930 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be075a3b-466e-4f38-8bbf-92a0fd4b0153 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:31.311548 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be075a3b-466e-4f38-8bbf-92a0fd4b0153 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:31.311822 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be075a3b-466e-4f38-8bbf-92a0fd4b0153 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:31.312127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be075a3b-466e-4f38-8bbf-92a0fd4b0153 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:31.321288 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be075a3b-466e-4f38-8bbf-92a0fd4b0153 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:31.322721 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be075a3b-466e-4f38-8bbf-92a0fd4b0153 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:31.332336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1084/2191] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:49:31 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 09:49:32.200820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc59b03-805a-421c-a23e-8c24e3c11d1c None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:32.202446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-edc59b03-805a-421c-a23e-8c24e3c11d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:32.272562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc59b03-805a-421c-a23e-8c24e3c11d1c None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:32.278178 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65597265-95fe-45b8-9a10-020ca1ab202c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:32.279061 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65597265-95fe-45b8-9a10-020ca1ab202c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:32.279337 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65597265-95fe-45b8-9a10-020ca1ab202c admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:32.279676 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65597265-95fe-45b8-9a10-020ca1ab202c admin admin] PATH_INFO: `/v3/users/2096c28d3dba4306a6e0059e137b949b` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:32.280922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65597265-95fe-45b8-9a10-020ca1ab202c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2096c28d3dba4306a6e0059e137b949b)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:32.283084 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65597265-95fe-45b8-9a10-020ca1ab202c admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:32.337381 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-65597265-95fe-45b8-9a10-020ca1ab202c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2096c28d3dba4306a6e0059e137b949b'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:32.344656 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-65597265-95fe-45b8-9a10-020ca1ab202c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2096c28d3dba4306a6e0059e137b949b'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:32.345617 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-65597265-95fe-45b8-9a10-020ca1ab202c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2096c28d3dba4306a6e0059e137b949b'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:32.354740 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-65597265-95fe-45b8-9a10-020ca1ab202c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2096c28d3dba4306a6e0059e137b949b'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:32.355703 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-65597265-95fe-45b8-9a10-020ca1ab202c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2096c28d3dba4306a6e0059e137b949b'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:32.375091 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1108/2192] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:49:32 2022] DELETE /identity/v3/users/2096c28d3dba4306a6e0059e137b949b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:49:32.379906 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7523168-a837-46d2-a0ad-4f7d0825f471 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:32.380446 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7523168-a837-46d2-a0ad-4f7d0825f471 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:32.380446 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7523168-a837-46d2-a0ad-4f7d0825f471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:32.399884 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-f7523168-a837-46d2-a0ad-4f7d0825f471 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:32.401165 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f7523168-a837-46d2-a0ad-4f7d0825f471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:32.464539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1085/2193] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:49:32.471422 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65597265-95fe-45b8-9a10-020ca1ab202c admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:32.473171 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-65597265-95fe-45b8-9a10-020ca1ab202c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:32.531825 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65597265-95fe-45b8-9a10-020ca1ab202c admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:32.536187 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3bd23-f15b-49b2-b5e8-04f70f5ce0ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:32.536892 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3bd23-f15b-49b2-b5e8-04f70f5ce0ed service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:32.537193 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3bd23-f15b-49b2-b5e8-04f70f5ce0ed service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:32.537548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3bd23-f15b-49b2-b5e8-04f70f5ce0ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:32.552708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba3bd23-f15b-49b2-b5e8-04f70f5ce0ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:32.552995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba3bd23-f15b-49b2-b5e8-04f70f5ce0ed service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:32.566549 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1109/2194] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:49:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:49:32.857106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7523168-a837-46d2-a0ad-4f7d0825f471 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:32.859101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f7523168-a837-46d2-a0ad-4f7d0825f471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:32.889212 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7523168-a837-46d2-a0ad-4f7d0825f471 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:32.900799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6fe17-787e-429e-a67b-5d432e5cdd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:32.902186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6fe17-787e-429e-a67b-5d432e5cdd0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:32.902652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6fe17-787e-429e-a67b-5d432e5cdd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:32.903093 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6fe17-787e-429e-a67b-5d432e5cdd0e admin admin] PATH_INFO: `/v3/projects/d5f05a4b0e2f496992c70c31b4aa26bc` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:32.904992 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6fe17-787e-429e-a67b-5d432e5cdd0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5f05a4b0e2f496992c70c31b4aa26bc)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:32.909561 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6fe17-787e-429e-a67b-5d432e5cdd0e admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:32.976376 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e0c6fe17-787e-429e-a67b-5d432e5cdd0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5f05a4b0e2f496992c70c31b4aa26bc', 'request_id': 'req-e0c6fe17-787e-429e-a67b-5d432e5cdd0e'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:32.982595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e0c6fe17-787e-429e-a67b-5d432e5cdd0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5f05a4b0e2f496992c70c31b4aa26bc', 'request_id': 'req-e0c6fe17-787e-429e-a67b-5d432e5cdd0e'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:32.988486 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e0c6fe17-787e-429e-a67b-5d432e5cdd0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5f05a4b0e2f496992c70c31b4aa26bc', 'request_id': 'req-e0c6fe17-787e-429e-a67b-5d432e5cdd0e'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:33.002339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e0c6fe17-787e-429e-a67b-5d432e5cdd0e admin admin] The token cache is being invalidate because project d5f05a4b0e2f496992c70c31b4aa26bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:49:33.002623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e0c6fe17-787e-429e-a67b-5d432e5cdd0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:33.003268 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e0c6fe17-787e-429e-a67b-5d432e5cdd0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:33.005199 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1086/2195] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:49:32 2022] DELETE /identity/v3/projects/d5f05a4b0e2f496992c70c31b4aa26bc => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:49:33.010127 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7362c18-24b4-47b8-96c6-01e9fe833ddd None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:33.010487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7362c18-24b4-47b8-96c6-01e9fe833ddd None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:33.010783 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7362c18-24b4-47b8-96c6-01e9fe833ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:33.030354 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-a7362c18-24b4-47b8-96c6-01e9fe833ddd None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:33.031783 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-a7362c18-24b4-47b8-96c6-01e9fe833ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:33.092087 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1110/2196] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:49:33.099958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdb0cd-166d-40c9-ac0f-feadaaeaf476 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:33.100237 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdb0cd-166d-40c9-ac0f-feadaaeaf476 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:33.100506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdb0cd-166d-40c9-ac0f-feadaaeaf476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:33.119533 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-5fcdb0cd-166d-40c9-ac0f-feadaaeaf476 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:33.120791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-5fcdb0cd-166d-40c9-ac0f-feadaaeaf476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:33.129178 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1087/2197] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:49:33.133248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7362c18-24b4-47b8-96c6-01e9fe833ddd None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:33.143245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7362c18-24b4-47b8-96c6-01e9fe833ddd None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:33.148435 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846278c-de09-4f34-9666-98ae49f651a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:33.149119 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846278c-de09-4f34-9666-98ae49f651a7 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:33.149400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846278c-de09-4f34-9666-98ae49f651a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:33.149670 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846278c-de09-4f34-9666-98ae49f651a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:33.150400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9846278c-de09-4f34-9666-98ae49f651a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:49:33.150705 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9846278c-de09-4f34-9666-98ae49f651a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:33.152332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9846278c-de09-4f34-9666-98ae49f651a7 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:33.163629 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1111/2198] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:49:33.167930 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcdb0cd-166d-40c9-ac0f-feadaaeaf476 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:33.177341 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcdb0cd-166d-40c9-ac0f-feadaaeaf476 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:33.183251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e89d6-6747-4b1f-b834-73ddccdfc027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:33.183896 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e89d6-6747-4b1f-b834-73ddccdfc027 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:33.184168 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e89d6-6747-4b1f-b834-73ddccdfc027 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:33.184434 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e89d6-6747-4b1f-b834-73ddccdfc027 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:33.185180 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e89d6-6747-4b1f-b834-73ddccdfc027 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:33.187074 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e89d6-6747-4b1f-b834-73ddccdfc027 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:33.218250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1088/2199] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:49:33 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:49:33.220432 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846278c-de09-4f34-9666-98ae49f651a7 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:33.231481 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846278c-de09-4f34-9666-98ae49f651a7 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:33.289986 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0469a92-e0a2-41e4-816f-6b41746fae39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:33.290651 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0469a92-e0a2-41e4-816f-6b41746fae39 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:33.290935 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0469a92-e0a2-41e4-816f-6b41746fae39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:33.291196 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0469a92-e0a2-41e4-816f-6b41746fae39 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:33.291903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0469a92-e0a2-41e4-816f-6b41746fae39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:33.293871 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0469a92-e0a2-41e4-816f-6b41746fae39 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:33.315047 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1112/2200] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:49:33 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:49:33.319362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e89d6-6747-4b1f-b834-73ddccdfc027 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:33.329825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e89d6-6747-4b1f-b834-73ddccdfc027 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:33.335502 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68216788-e38a-4a40-98a0-c59b4bff83b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:33.336292 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68216788-e38a-4a40-98a0-c59b4bff83b7 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:33.336606 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68216788-e38a-4a40-98a0-c59b4bff83b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:33.336966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68216788-e38a-4a40-98a0-c59b4bff83b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:33.337755 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68216788-e38a-4a40-98a0-c59b4bff83b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:33.339421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68216788-e38a-4a40-98a0-c59b4bff83b7 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:33.351076 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1089/2201] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:49:33.355086 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0469a92-e0a2-41e4-816f-6b41746fae39 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:33.365477 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0469a92-e0a2-41e4-816f-6b41746fae39 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:33.370531 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b011bfb-aacc-4830-b241-0d0cbd3b88f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:33.371188 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b011bfb-aacc-4830-b241-0d0cbd3b88f7 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:33.371468 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b011bfb-aacc-4830-b241-0d0cbd3b88f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:33.371735 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b011bfb-aacc-4830-b241-0d0cbd3b88f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:33.372479 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b011bfb-aacc-4830-b241-0d0cbd3b88f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:33.374277 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b011bfb-aacc-4830-b241-0d0cbd3b88f7 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:33.385279 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1113/2202] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:49:33.389645 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68216788-e38a-4a40-98a0-c59b4bff83b7 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:33.400481 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68216788-e38a-4a40-98a0-c59b4bff83b7 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:33.405687 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae534300-c3ee-4807-8220-53d9ef9557c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:33.406564 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae534300-c3ee-4807-8220-53d9ef9557c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:33.406839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae534300-c3ee-4807-8220-53d9ef9557c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:33.407094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae534300-c3ee-4807-8220-53d9ef9557c1 admin admin] PATH_INFO: `/v3/projects/190c08f1683e4593b0518c93d639be4c/users/db3a5745a11e41b4adcea40e47e6a83b/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:33.423021 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae534300-c3ee-4807-8220-53d9ef9557c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=190c08f1683e4593b0518c93d639be4c, user_id=db3a5745a11e41b4adcea40e47e6a83b, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:33.425772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae534300-c3ee-4807-8220-53d9ef9557c1 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:33.446858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1090/2203] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:49:33 2022] PUT /identity/v3/projects/190c08f1683e4593b0518c93d639be4c/users/db3a5745a11e41b4adcea40e47e6a83b/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:49:33.459019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05345b6-73d8-4d35-8595-37dcc2268e77 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:33.459587 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05345b6-73d8-4d35-8595-37dcc2268e77 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:33.460241 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05345b6-73d8-4d35-8595-37dcc2268e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:33.495130 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-d05345b6-73d8-4d35-8595-37dcc2268e77 None None] MFA Rules not processed for user `db3a5745a11e41b4adcea40e47e6a83b`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:33.496465 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-d05345b6-73d8-4d35-8595-37dcc2268e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:33.575441 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1114/2204] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:49:33.586615 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae534300-c3ee-4807-8220-53d9ef9557c1 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:33.588311 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-ae534300-c3ee-4807-8220-53d9ef9557c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:33.656292 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae534300-c3ee-4807-8220-53d9ef9557c1 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:33.662374 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f85a75-8d72-4197-aac8-133d05a309d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:33.663047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f85a75-8d72-4197-aac8-133d05a309d1 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:33.663313 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f85a75-8d72-4197-aac8-133d05a309d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:33.663584 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f85a75-8d72-4197-aac8-133d05a309d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:33.675604 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f85a75-8d72-4197-aac8-133d05a309d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:33.677275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f85a75-8d72-4197-aac8-133d05a309d1 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:33.686717 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1091/2205] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:49:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:49:33.714401 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05345b6-73d8-4d35-8595-37dcc2268e77 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:33.775104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05345b6-73d8-4d35-8595-37dcc2268e77 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:33.779956 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6505909-24c7-4383-83cd-9c3884144c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:33.780878 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6505909-24c7-4383-83cd-9c3884144c03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:33.781160 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6505909-24c7-4383-83cd-9c3884144c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:33.781447 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6505909-24c7-4383-83cd-9c3884144c03 admin admin] PATH_INFO: `/v3/users/db3a5745a11e41b4adcea40e47e6a83b` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:33.782766 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6505909-24c7-4383-83cd-9c3884144c03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db3a5745a11e41b4adcea40e47e6a83b)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:33.785246 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6505909-24c7-4383-83cd-9c3884144c03 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:33.835531 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e6505909-24c7-4383-83cd-9c3884144c03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db3a5745a11e41b4adcea40e47e6a83b'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:33.842449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e6505909-24c7-4383-83cd-9c3884144c03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db3a5745a11e41b4adcea40e47e6a83b'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:33.843147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e6505909-24c7-4383-83cd-9c3884144c03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db3a5745a11e41b4adcea40e47e6a83b'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:33.851825 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e6505909-24c7-4383-83cd-9c3884144c03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db3a5745a11e41b4adcea40e47e6a83b'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:33.852758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e6505909-24c7-4383-83cd-9c3884144c03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db3a5745a11e41b4adcea40e47e6a83b'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:33.874774 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1115/2206] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:49:33 2022] DELETE /identity/v3/users/db3a5745a11e41b4adcea40e47e6a83b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:49:33.879360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5b030-bd2e-418d-9dcb-f7705a7ce9a4 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:33.879636 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5b030-bd2e-418d-9dcb-f7705a7ce9a4 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:33.879910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5b030-bd2e-418d-9dcb-f7705a7ce9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:33.903043 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-78b5b030-bd2e-418d-9dcb-f7705a7ce9a4 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:33.904309 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-78b5b030-bd2e-418d-9dcb-f7705a7ce9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:33.966116 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1092/2207] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:49:33.976392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6505909-24c7-4383-83cd-9c3884144c03 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:33.978163 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e6505909-24c7-4383-83cd-9c3884144c03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:34.038756 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6505909-24c7-4383-83cd-9c3884144c03 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:34.042694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a5e50-2759-49b8-b942-abe9d5ccf7e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:34.043443 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a5e50-2759-49b8-b942-abe9d5ccf7e8 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:34.043830 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a5e50-2759-49b8-b942-abe9d5ccf7e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:34.044269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a5e50-2759-49b8-b942-abe9d5ccf7e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:34.055447 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a5e50-2759-49b8-b942-abe9d5ccf7e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:34.057325 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a5e50-2759-49b8-b942-abe9d5ccf7e8 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:34.067925 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1116/2208] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:49:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:49:34.472079 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5b030-bd2e-418d-9dcb-f7705a7ce9a4 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:34.474094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-78b5b030-bd2e-418d-9dcb-f7705a7ce9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:34.493516 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5b030-bd2e-418d-9dcb-f7705a7ce9a4 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:34.499601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76383698-a338-4ac1-9b92-571a4ece313e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:34.500700 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76383698-a338-4ac1-9b92-571a4ece313e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:34.501046 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76383698-a338-4ac1-9b92-571a4ece313e admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:34.501374 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76383698-a338-4ac1-9b92-571a4ece313e admin admin] PATH_INFO: `/v3/projects/190c08f1683e4593b0518c93d639be4c` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:34.502692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76383698-a338-4ac1-9b92-571a4ece313e admin admin] RBAC: Authorizing `identity:delete_project(project_id=190c08f1683e4593b0518c93d639be4c)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:34.504802 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76383698-a338-4ac1-9b92-571a4ece313e admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:34.555440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-76383698-a338-4ac1-9b92-571a4ece313e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '190c08f1683e4593b0518c93d639be4c', 'request_id': 'req-76383698-a338-4ac1-9b92-571a4ece313e'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:34.562600 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-76383698-a338-4ac1-9b92-571a4ece313e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '190c08f1683e4593b0518c93d639be4c', 'request_id': 'req-76383698-a338-4ac1-9b92-571a4ece313e'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:34.568955 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-76383698-a338-4ac1-9b92-571a4ece313e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '190c08f1683e4593b0518c93d639be4c', 'request_id': 'req-76383698-a338-4ac1-9b92-571a4ece313e'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:34.588089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-76383698-a338-4ac1-9b92-571a4ece313e admin admin] The token cache is being invalidate because project 190c08f1683e4593b0518c93d639be4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:49:34.588428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-76383698-a338-4ac1-9b92-571a4ece313e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:34.589428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-76383698-a338-4ac1-9b92-571a4ece313e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:34.592701 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1093/2209] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:49:34 2022] DELETE /identity/v3/projects/190c08f1683e4593b0518c93d639be4c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:49:42.948899 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a5e50-2759-49b8-b942-abe9d5ccf7e8 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:42.950626 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-615a5e50-2759-49b8-b942-abe9d5ccf7e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:42.964268 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a5e50-2759-49b8-b942-abe9d5ccf7e8 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:43.020452 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e006b4-6d86-40a4-b805-b5cf765ebd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:43.021348 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e006b4-6d86-40a4-b805-b5cf765ebd96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:43.021625 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e006b4-6d86-40a4-b805-b5cf765ebd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:43.021899 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e006b4-6d86-40a4-b805-b5cf765ebd96 admin admin] PATH_INFO: `/v3/users/085db625954b4869a7f81ce2eb0416f0` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:43.023092 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e006b4-6d86-40a4-b805-b5cf765ebd96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=085db625954b4869a7f81ce2eb0416f0)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:43.025086 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e006b4-6d86-40a4-b805-b5cf765ebd96 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:43.072039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-06e006b4-6d86-40a4-b805-b5cf765ebd96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '085db625954b4869a7f81ce2eb0416f0'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:43.079066 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-06e006b4-6d86-40a4-b805-b5cf765ebd96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '085db625954b4869a7f81ce2eb0416f0'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:43.079741 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-06e006b4-6d86-40a4-b805-b5cf765ebd96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '085db625954b4869a7f81ce2eb0416f0'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:43.088824 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-06e006b4-6d86-40a4-b805-b5cf765ebd96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '085db625954b4869a7f81ce2eb0416f0'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:43.089524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-06e006b4-6d86-40a4-b805-b5cf765ebd96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '085db625954b4869a7f81ce2eb0416f0'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:43.107538 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1117/2210] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:49:42 2022] DELETE /identity/v3/users/085db625954b4869a7f81ce2eb0416f0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:49:43.112612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b104fbb-4a6d-4b88-aa99-ca99a7b3a115 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:43.113093 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b104fbb-4a6d-4b88-aa99-ca99a7b3a115 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:43.113187 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b104fbb-4a6d-4b88-aa99-ca99a7b3a115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:43.132609 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-1b104fbb-4a6d-4b88-aa99-ca99a7b3a115 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:43.133978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-1b104fbb-4a6d-4b88-aa99-ca99a7b3a115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:43.191663 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1094/2211] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:49:43.200979 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e006b4-6d86-40a4-b805-b5cf765ebd96 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:43.202525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-06e006b4-6d86-40a4-b805-b5cf765ebd96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:43.258894 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e006b4-6d86-40a4-b805-b5cf765ebd96 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:43.262863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a5c72-81f6-4506-8afc-812ecfe96fbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:43.263493 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a5c72-81f6-4506-8afc-812ecfe96fbc service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:43.263795 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a5c72-81f6-4506-8afc-812ecfe96fbc service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:43.264075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a5c72-81f6-4506-8afc-812ecfe96fbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:43.273260 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a5c72-81f6-4506-8afc-812ecfe96fbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:43.274698 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a5c72-81f6-4506-8afc-812ecfe96fbc service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:43.285755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1118/2212] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:49:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:49:43.556620 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b104fbb-4a6d-4b88-aa99-ca99a7b3a115 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:43.558179 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-1b104fbb-4a6d-4b88-aa99-ca99a7b3a115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:43.572077 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b104fbb-4a6d-4b88-aa99-ca99a7b3a115 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:43.578143 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0493c4-7e18-4990-9e3e-163149dae05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:43.579011 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0493c4-7e18-4990-9e3e-163149dae05e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:43.579279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0493c4-7e18-4990-9e3e-163149dae05e admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:43.579632 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0493c4-7e18-4990-9e3e-163149dae05e admin admin] PATH_INFO: `/v3/projects/91744b9cc5b14da1949db5dc161b433c` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:43.581795 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0493c4-7e18-4990-9e3e-163149dae05e admin admin] RBAC: Authorizing `identity:delete_project(project_id=91744b9cc5b14da1949db5dc161b433c)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:43.584080 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0493c4-7e18-4990-9e3e-163149dae05e admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:43.634733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0f0493c4-7e18-4990-9e3e-163149dae05e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91744b9cc5b14da1949db5dc161b433c', 'request_id': 'req-0f0493c4-7e18-4990-9e3e-163149dae05e'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:43.640908 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0f0493c4-7e18-4990-9e3e-163149dae05e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91744b9cc5b14da1949db5dc161b433c', 'request_id': 'req-0f0493c4-7e18-4990-9e3e-163149dae05e'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:43.646332 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0f0493c4-7e18-4990-9e3e-163149dae05e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91744b9cc5b14da1949db5dc161b433c', 'request_id': 'req-0f0493c4-7e18-4990-9e3e-163149dae05e'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:43.659782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0f0493c4-7e18-4990-9e3e-163149dae05e admin admin] The token cache is being invalidate because project 91744b9cc5b14da1949db5dc161b433c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:49:43.660057 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0f0493c4-7e18-4990-9e3e-163149dae05e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:43.660649 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0f0493c4-7e18-4990-9e3e-163149dae05e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:43.662220 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1095/2213] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:49:43 2022] DELETE /identity/v3/projects/91744b9cc5b14da1949db5dc161b433c => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:49:43.667298 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6dcd2-2f14-42de-8dc8-a854d0054991 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:43.667637 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6dcd2-2f14-42de-8dc8-a854d0054991 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:43.667910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6dcd2-2f14-42de-8dc8-a854d0054991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:43.697250 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-74f6dcd2-2f14-42de-8dc8-a854d0054991 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:43.698482 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-74f6dcd2-2f14-42de-8dc8-a854d0054991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:43.764110 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1119/2214] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:49:43.772238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81afd9-5281-49b4-b303-9f4935c8a3d2 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:43.772499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81afd9-5281-49b4-b303-9f4935c8a3d2 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:43.772830 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81afd9-5281-49b4-b303-9f4935c8a3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:43.791590 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-2f81afd9-5281-49b4-b303-9f4935c8a3d2 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:43.792842 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-2f81afd9-5281-49b4-b303-9f4935c8a3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:43.800913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1096/2215] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:49:43.805424 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6dcd2-2f14-42de-8dc8-a854d0054991 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:43.815365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6dcd2-2f14-42de-8dc8-a854d0054991 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:43.820329 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6683093-624c-4bd1-89aa-b1ac9f32a8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:43.821011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6683093-624c-4bd1-89aa-b1ac9f32a8e1 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:43.821295 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6683093-624c-4bd1-89aa-b1ac9f32a8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:43.821555 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6683093-624c-4bd1-89aa-b1ac9f32a8e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:43.822561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6683093-624c-4bd1-89aa-b1ac9f32a8e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:49:43.822899 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6683093-624c-4bd1-89aa-b1ac9f32a8e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:43.824521 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6683093-624c-4bd1-89aa-b1ac9f32a8e1 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:43.840343 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1120/2216] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:49:43.841621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81afd9-5281-49b4-b303-9f4935c8a3d2 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:43.851184 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81afd9-5281-49b4-b303-9f4935c8a3d2 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:43.857060 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069da3e5-3e81-4eab-a813-c3b75cb64d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:43.857725 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069da3e5-3e81-4eab-a813-c3b75cb64d32 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:43.858008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069da3e5-3e81-4eab-a813-c3b75cb64d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:43.858624 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069da3e5-3e81-4eab-a813-c3b75cb64d32 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:43.859316 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069da3e5-3e81-4eab-a813-c3b75cb64d32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:43.861277 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069da3e5-3e81-4eab-a813-c3b75cb64d32 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:43.891144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1097/2217] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:49:43 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:49:43.895602 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6683093-624c-4bd1-89aa-b1ac9f32a8e1 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:43.907266 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6683093-624c-4bd1-89aa-b1ac9f32a8e1 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:43.971588 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28c209-ec10-4619-bb2c-d62e930db3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:43.973666 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28c209-ec10-4619-bb2c-d62e930db3f1 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:43.973946 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28c209-ec10-4619-bb2c-d62e930db3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:43.974219 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28c209-ec10-4619-bb2c-d62e930db3f1 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:43.974973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28c209-ec10-4619-bb2c-d62e930db3f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:43.977101 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28c209-ec10-4619-bb2c-d62e930db3f1 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:44.006211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1121/2218] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:49:43 2022] POST /identity/v3/users => generated 282 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:49:44.007642 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069da3e5-3e81-4eab-a813-c3b75cb64d32 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:44.017787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069da3e5-3e81-4eab-a813-c3b75cb64d32 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:44.022604 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce603b-6408-473c-bd4e-c8769ad817f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:44.023290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce603b-6408-473c-bd4e-c8769ad817f8 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:44.023589 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce603b-6408-473c-bd4e-c8769ad817f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:44.023863 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce603b-6408-473c-bd4e-c8769ad817f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:44.024593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce603b-6408-473c-bd4e-c8769ad817f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:44.026267 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce603b-6408-473c-bd4e-c8769ad817f8 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:44.039723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1098/2219] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:49:44.041194 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28c209-ec10-4619-bb2c-d62e930db3f1 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:44.051305 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28c209-ec10-4619-bb2c-d62e930db3f1 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:44.056015 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d048c-431b-4b41-b599-372766207554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:44.056711 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d048c-431b-4b41-b599-372766207554 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:44.057115 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d048c-431b-4b41-b599-372766207554 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:44.057421 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d048c-431b-4b41-b599-372766207554 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:44.058150 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d048c-431b-4b41-b599-372766207554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:44.059863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d048c-431b-4b41-b599-372766207554 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:44.069808 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1122/2220] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:49:44.074169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce603b-6408-473c-bd4e-c8769ad817f8 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:44.084548 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce603b-6408-473c-bd4e-c8769ad817f8 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:44.089548 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def781e-bb36-4cf7-9363-dc22de9d0103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:44.090429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def781e-bb36-4cf7-9363-dc22de9d0103 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:44.090703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def781e-bb36-4cf7-9363-dc22de9d0103 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:44.090968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def781e-bb36-4cf7-9363-dc22de9d0103 admin admin] PATH_INFO: `/v3/projects/2f8d44416aa54322a07fd60309982fe7/users/8f7a46773d124e1eb0888e728a3a70d2/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:44.105882 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def781e-bb36-4cf7-9363-dc22de9d0103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f8d44416aa54322a07fd60309982fe7, user_id=8f7a46773d124e1eb0888e728a3a70d2, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:44.108508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def781e-bb36-4cf7-9363-dc22de9d0103 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:44.122965 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1099/2221] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:49:44 2022] PUT /identity/v3/projects/2f8d44416aa54322a07fd60309982fe7/users/8f7a46773d124e1eb0888e728a3a70d2/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:49:44.133660 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef4a2e-08b7-4ba8-b372-b7bece3319da None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:44.133946 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef4a2e-08b7-4ba8-b372-b7bece3319da None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:44.134217 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef4a2e-08b7-4ba8-b372-b7bece3319da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:44.159352 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-a5ef4a2e-08b7-4ba8-b372-b7bece3319da None None] MFA Rules not processed for user `8f7a46773d124e1eb0888e728a3a70d2`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:44.160589 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-a5ef4a2e-08b7-4ba8-b372-b7bece3319da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:44.219605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1123/2222] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:44 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:49:44.224521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def781e-bb36-4cf7-9363-dc22de9d0103 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:44.293501 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def781e-bb36-4cf7-9363-dc22de9d0103 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:44.299030 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3269f-28a7-49d8-8eba-9a3762e06dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:44.299705 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3269f-28a7-49d8-8eba-9a3762e06dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:44.299993 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3269f-28a7-49d8-8eba-9a3762e06dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:44.300266 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3269f-28a7-49d8-8eba-9a3762e06dd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:44.300996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3269f-28a7-49d8-8eba-9a3762e06dd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:44.302982 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3269f-28a7-49d8-8eba-9a3762e06dd5 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:44.341535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1100/2223] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:49:44 2022] POST /identity/v3/projects => generated 363 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:49:44.346335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef4a2e-08b7-4ba8-b372-b7bece3319da None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:44.360399 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef4a2e-08b7-4ba8-b372-b7bece3319da None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:44.423769 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848765d8-59ea-4b70-afcb-9cab5c6b5e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:44.424423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848765d8-59ea-4b70-afcb-9cab5c6b5e45 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:44.424735 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848765d8-59ea-4b70-afcb-9cab5c6b5e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:44.425005 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848765d8-59ea-4b70-afcb-9cab5c6b5e45 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:44.425701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848765d8-59ea-4b70-afcb-9cab5c6b5e45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:44.427684 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848765d8-59ea-4b70-afcb-9cab5c6b5e45 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:44.450813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1124/2224] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:49:44 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:49:44.455086 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3269f-28a7-49d8-8eba-9a3762e06dd5 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:44.465984 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3269f-28a7-49d8-8eba-9a3762e06dd5 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:44.470897 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc05464-3329-4d74-86f6-b99f579570a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:44.471576 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc05464-3329-4d74-86f6-b99f579570a4 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:44.471849 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc05464-3329-4d74-86f6-b99f579570a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:44.472118 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc05464-3329-4d74-86f6-b99f579570a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:44.472947 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc05464-3329-4d74-86f6-b99f579570a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:44.474539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc05464-3329-4d74-86f6-b99f579570a4 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:44.488742 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1101/2225] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:49:44.489996 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848765d8-59ea-4b70-afcb-9cab5c6b5e45 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:44.499929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848765d8-59ea-4b70-afcb-9cab5c6b5e45 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:44.505020 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d1d18-f603-4174-ab39-5cb7de267e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:44.505903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d1d18-f603-4174-ab39-5cb7de267e9e admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:44.506200 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d1d18-f603-4174-ab39-5cb7de267e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:44.506461 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d1d18-f603-4174-ab39-5cb7de267e9e admin admin] PATH_INFO: `/v3/projects/c4d2555012164a83892bc9e8009df2f3/users/fc325ce5f3884bf482034c84b3110b85/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:44.520468 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d1d18-f603-4174-ab39-5cb7de267e9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4d2555012164a83892bc9e8009df2f3, user_id=fc325ce5f3884bf482034c84b3110b85, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:44.523188 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d1d18-f603-4174-ab39-5cb7de267e9e admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:44.536805 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1125/2226] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:49:44 2022] PUT /identity/v3/projects/c4d2555012164a83892bc9e8009df2f3/users/fc325ce5f3884bf482034c84b3110b85/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:49:44.542254 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc05464-3329-4d74-86f6-b99f579570a4 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:44.552431 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc05464-3329-4d74-86f6-b99f579570a4 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:44.607407 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9fdc2-d13f-4ab4-af02-17a1d3f0963d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:44.608021 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9fdc2-d13f-4ab4-af02-17a1d3f0963d admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:44.608821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9fdc2-d13f-4ab4-af02-17a1d3f0963d admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:44.608997 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9fdc2-d13f-4ab4-af02-17a1d3f0963d admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:44.609666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9fdc2-d13f-4ab4-af02-17a1d3f0963d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:44.611286 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9fdc2-d13f-4ab4-af02-17a1d3f0963d admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:44.621211 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1102/2227] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:49:44.625684 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d1d18-f603-4174-ab39-5cb7de267e9e admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:44.636618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d1d18-f603-4174-ab39-5cb7de267e9e admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:44.642127 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea00967-ff06-4193-a6c5-d03e8cfc4b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:44.643030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea00967-ff06-4193-a6c5-d03e8cfc4b0f admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:44.643307 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea00967-ff06-4193-a6c5-d03e8cfc4b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:44.643577 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea00967-ff06-4193-a6c5-d03e8cfc4b0f admin admin] PATH_INFO: `/v3/projects/c4d2555012164a83892bc9e8009df2f3/users/fc325ce5f3884bf482034c84b3110b85/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:44.645812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea00967-ff06-4193-a6c5-d03e8cfc4b0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4d2555012164a83892bc9e8009df2f3, user_id=fc325ce5f3884bf482034c84b3110b85, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:44.648468 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea00967-ff06-4193-a6c5-d03e8cfc4b0f admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:44.659264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1126/2228] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:49:44 2022] PUT /identity/v3/projects/c4d2555012164a83892bc9e8009df2f3/users/fc325ce5f3884bf482034c84b3110b85/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:49:44.670758 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271b0d8-21a2-41cb-a549-6135e0ddb4aa None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:44.671032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271b0d8-21a2-41cb-a549-6135e0ddb4aa None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:44.671301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271b0d8-21a2-41cb-a549-6135e0ddb4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:44.697525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-b271b0d8-21a2-41cb-a549-6135e0ddb4aa None None] MFA Rules not processed for user `fc325ce5f3884bf482034c84b3110b85`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:44.698718 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-b271b0d8-21a2-41cb-a549-6135e0ddb4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:44.770626 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1103/2229] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:44 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:49:44.787573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea00967-ff06-4193-a6c5-d03e8cfc4b0f admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:44.789727 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-fea00967-ff06-4193-a6c5-d03e8cfc4b0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:44.860423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea00967-ff06-4193-a6c5-d03e8cfc4b0f admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:44.866124 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c6ea9-413f-4616-bd39-e7c9a7dbdc92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:44.866768 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c6ea9-413f-4616-bd39-e7c9a7dbdc92 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:44.867046 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c6ea9-413f-4616-bd39-e7c9a7dbdc92 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:44.867314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c6ea9-413f-4616-bd39-e7c9a7dbdc92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:44.877539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c6ea9-413f-4616-bd39-e7c9a7dbdc92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:44.878987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c6ea9-413f-4616-bd39-e7c9a7dbdc92 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:44.888821 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1127/2230] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:49:44 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:49:45.306184 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271b0d8-21a2-41cb-a549-6135e0ddb4aa None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:45.365134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271b0d8-21a2-41cb-a549-6135e0ddb4aa None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:45.370197 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8fe3f2-0045-4f1d-8025-b8308e3fb990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:45.371054 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fe3f2-0045-4f1d-8025-b8308e3fb990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:45.371329 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fe3f2-0045-4f1d-8025-b8308e3fb990 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:45.371596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fe3f2-0045-4f1d-8025-b8308e3fb990 admin admin] PATH_INFO: `/v3/users/8f7a46773d124e1eb0888e728a3a70d2` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:45.372918 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8fe3f2-0045-4f1d-8025-b8308e3fb990 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f7a46773d124e1eb0888e728a3a70d2)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:45.374844 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8fe3f2-0045-4f1d-8025-b8308e3fb990 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:45.434258 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6b8fe3f2-0045-4f1d-8025-b8308e3fb990 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f7a46773d124e1eb0888e728a3a70d2'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:45.441285 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6b8fe3f2-0045-4f1d-8025-b8308e3fb990 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f7a46773d124e1eb0888e728a3a70d2'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:45.450429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6b8fe3f2-0045-4f1d-8025-b8308e3fb990 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f7a46773d124e1eb0888e728a3a70d2'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:45.451112 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6b8fe3f2-0045-4f1d-8025-b8308e3fb990 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f7a46773d124e1eb0888e728a3a70d2'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:45.451822 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6b8fe3f2-0045-4f1d-8025-b8308e3fb990 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f7a46773d124e1eb0888e728a3a70d2'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:45.469297 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1104/2231] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:49:45 2022] DELETE /identity/v3/users/8f7a46773d124e1eb0888e728a3a70d2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:49:45.474153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263832c1-3534-4f56-bbd6-e01c5cefec13 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:45.474618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263832c1-3534-4f56-bbd6-e01c5cefec13 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:45.474730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263832c1-3534-4f56-bbd6-e01c5cefec13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:45.492527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-263832c1-3534-4f56-bbd6-e01c5cefec13 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:45.493755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-263832c1-3534-4f56-bbd6-e01c5cefec13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:45.557675 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1128/2232] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:49:45.567723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8fe3f2-0045-4f1d-8025-b8308e3fb990 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:45.569277 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-6b8fe3f2-0045-4f1d-8025-b8308e3fb990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:45.637523 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8fe3f2-0045-4f1d-8025-b8308e3fb990 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:45.641748 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39061de-c99f-4608-9a17-9a3991129175 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:45.642384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39061de-c99f-4608-9a17-9a3991129175 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:45.642657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39061de-c99f-4608-9a17-9a3991129175 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:45.642922 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39061de-c99f-4608-9a17-9a3991129175 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:45.653333 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39061de-c99f-4608-9a17-9a3991129175 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:45.655604 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39061de-c99f-4608-9a17-9a3991129175 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:45.671036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1105/2233] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:49:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:49:45.943468 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263832c1-3534-4f56-bbd6-e01c5cefec13 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:45.944942 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-263832c1-3534-4f56-bbd6-e01c5cefec13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:45.957320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263832c1-3534-4f56-bbd6-e01c5cefec13 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:45.963117 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328b906-0f12-4861-a3d5-f5df83a07186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:45.963988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328b906-0f12-4861-a3d5-f5df83a07186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:45.964241 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328b906-0f12-4861-a3d5-f5df83a07186 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:45.964503 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328b906-0f12-4861-a3d5-f5df83a07186 admin admin] PATH_INFO: `/v3/projects/2f8d44416aa54322a07fd60309982fe7` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:45.965829 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4328b906-0f12-4861-a3d5-f5df83a07186 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f8d44416aa54322a07fd60309982fe7)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:45.967774 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4328b906-0f12-4861-a3d5-f5df83a07186 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:46.023293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-4328b906-0f12-4861-a3d5-f5df83a07186 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f8d44416aa54322a07fd60309982fe7', 'request_id': 'req-4328b906-0f12-4861-a3d5-f5df83a07186'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:46.032968 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-4328b906-0f12-4861-a3d5-f5df83a07186 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f8d44416aa54322a07fd60309982fe7', 'request_id': 'req-4328b906-0f12-4861-a3d5-f5df83a07186'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:46.040530 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-4328b906-0f12-4861-a3d5-f5df83a07186 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f8d44416aa54322a07fd60309982fe7', 'request_id': 'req-4328b906-0f12-4861-a3d5-f5df83a07186'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:46.055910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-4328b906-0f12-4861-a3d5-f5df83a07186 admin admin] The token cache is being invalidate because project 2f8d44416aa54322a07fd60309982fe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:49:46.056199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-4328b906-0f12-4861-a3d5-f5df83a07186 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:46.056877 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-4328b906-0f12-4861-a3d5-f5df83a07186 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:46.058687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1129/2234] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:49:45 2022] DELETE /identity/v3/projects/2f8d44416aa54322a07fd60309982fe7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:49:46.062849 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39061de-c99f-4608-9a17-9a3991129175 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:46.072822 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39061de-c99f-4608-9a17-9a3991129175 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:46.130609 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80037811-d4ab-43a4-8a95-d31920234a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:46.131532 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80037811-d4ab-43a4-8a95-d31920234a0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:46.131786 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80037811-d4ab-43a4-8a95-d31920234a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:46.132041 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80037811-d4ab-43a4-8a95-d31920234a0a admin admin] PATH_INFO: `/v3/users/fc325ce5f3884bf482034c84b3110b85` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:46.133361 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80037811-d4ab-43a4-8a95-d31920234a0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc325ce5f3884bf482034c84b3110b85)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:46.135336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80037811-d4ab-43a4-8a95-d31920234a0a admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:46.181188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-80037811-d4ab-43a4-8a95-d31920234a0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc325ce5f3884bf482034c84b3110b85'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:46.187976 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-80037811-d4ab-43a4-8a95-d31920234a0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc325ce5f3884bf482034c84b3110b85'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:46.196532 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-80037811-d4ab-43a4-8a95-d31920234a0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc325ce5f3884bf482034c84b3110b85'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:46.197181 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-80037811-d4ab-43a4-8a95-d31920234a0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc325ce5f3884bf482034c84b3110b85'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:46.197797 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-80037811-d4ab-43a4-8a95-d31920234a0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc325ce5f3884bf482034c84b3110b85'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:46.216023 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1106/2235] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:49:46 2022] DELETE /identity/v3/users/fc325ce5f3884bf482034c84b3110b85 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:49:46.579236 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328b906-0f12-4861-a3d5-f5df83a07186 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:46.580840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-4328b906-0f12-4861-a3d5-f5df83a07186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:46.602057 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328b906-0f12-4861-a3d5-f5df83a07186 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:46.671466 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1b30d-88fe-4d0e-b8f6-441a779a7aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:46.672327 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1b30d-88fe-4d0e-b8f6-441a779a7aed admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:46.672596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1b30d-88fe-4d0e-b8f6-441a779a7aed admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:46.672904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1b30d-88fe-4d0e-b8f6-441a779a7aed admin admin] PATH_INFO: `/v3/projects/c4d2555012164a83892bc9e8009df2f3` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:46.674161 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d1b30d-88fe-4d0e-b8f6-441a779a7aed admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4d2555012164a83892bc9e8009df2f3)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:46.676067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d1b30d-88fe-4d0e-b8f6-441a779a7aed admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:46.723391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-25d1b30d-88fe-4d0e-b8f6-441a779a7aed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4d2555012164a83892bc9e8009df2f3', 'request_id': 'req-25d1b30d-88fe-4d0e-b8f6-441a779a7aed'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:46.729587 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-25d1b30d-88fe-4d0e-b8f6-441a779a7aed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4d2555012164a83892bc9e8009df2f3', 'request_id': 'req-25d1b30d-88fe-4d0e-b8f6-441a779a7aed'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:46.735545 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-25d1b30d-88fe-4d0e-b8f6-441a779a7aed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4d2555012164a83892bc9e8009df2f3', 'request_id': 'req-25d1b30d-88fe-4d0e-b8f6-441a779a7aed'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:46.748982 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-25d1b30d-88fe-4d0e-b8f6-441a779a7aed admin admin] The token cache is being invalidate because project c4d2555012164a83892bc9e8009df2f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:49:46.749261 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-25d1b30d-88fe-4d0e-b8f6-441a779a7aed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:46.749879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-25d1b30d-88fe-4d0e-b8f6-441a779a7aed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:49:46.751426 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1130/2236] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:49:46 2022] DELETE /identity/v3/projects/c4d2555012164a83892bc9e8009df2f3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:49:46.757471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff6c68-5912-4786-96d6-4e8abcade627 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:46.757980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff6c68-5912-4786-96d6-4e8abcade627 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:46.758087 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff6c68-5912-4786-96d6-4e8abcade627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:46.785042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-2eff6c68-5912-4786-96d6-4e8abcade627 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:46.786401 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-2eff6c68-5912-4786-96d6-4e8abcade627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:46.863624 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1107/2237] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:49:46.871653 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c860b-5611-4507-8ff5-5ba0a400bd62 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:46.871934 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c860b-5611-4507-8ff5-5ba0a400bd62 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:46.872213 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c860b-5611-4507-8ff5-5ba0a400bd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:46.893604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-b96c860b-5611-4507-8ff5-5ba0a400bd62 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:46.896992 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b96c860b-5611-4507-8ff5-5ba0a400bd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:46.906700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1131/2238] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:49:46.912206 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff6c68-5912-4786-96d6-4e8abcade627 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:46.925315 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff6c68-5912-4786-96d6-4e8abcade627 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:46.931786 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7790e7-6730-4480-a257-1a801b1542a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:46.932440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7790e7-6730-4480-a257-1a801b1542a1 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:46.932739 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7790e7-6730-4480-a257-1a801b1542a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:46.933004 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7790e7-6730-4480-a257-1a801b1542a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:46.933724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7790e7-6730-4480-a257-1a801b1542a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:49:46.934056 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7790e7-6730-4480-a257-1a801b1542a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:46.936397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7790e7-6730-4480-a257-1a801b1542a1 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:46.953080 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1108/2239] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:49:46.959796 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c860b-5611-4507-8ff5-5ba0a400bd62 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:46.975505 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c860b-5611-4507-8ff5-5ba0a400bd62 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:46.985699 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b319c68f-6121-4e61-945a-47bebcbc7375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:46.986661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319c68f-6121-4e61-945a-47bebcbc7375 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:46.986930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319c68f-6121-4e61-945a-47bebcbc7375 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:46.987189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319c68f-6121-4e61-945a-47bebcbc7375 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:46.987871 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b319c68f-6121-4e61-945a-47bebcbc7375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:46.989954 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b319c68f-6121-4e61-945a-47bebcbc7375 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:47.019034 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1132/2240] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:49:46 2022] POST /identity/v3/projects => generated 367 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:49:47.023503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7790e7-6730-4480-a257-1a801b1542a1 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:47.034038 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7790e7-6730-4480-a257-1a801b1542a1 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:47.094922 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741c319-1b67-4897-a512-2eeae89b5fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:47.095595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741c319-1b67-4897-a512-2eeae89b5fb6 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:47.095989 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741c319-1b67-4897-a512-2eeae89b5fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:47.096259 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741c319-1b67-4897-a512-2eeae89b5fb6 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:47.096963 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7741c319-1b67-4897-a512-2eeae89b5fb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:47.099033 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7741c319-1b67-4897-a512-2eeae89b5fb6 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:47.122748 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1109/2241] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:49:47 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:49:47.126823 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b319c68f-6121-4e61-945a-47bebcbc7375 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:47.137349 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b319c68f-6121-4e61-945a-47bebcbc7375 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:47.142727 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7d788-05bc-4280-a894-5d9a71ef3be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:47.143401 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7d788-05bc-4280-a894-5d9a71ef3be2 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:47.143679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7d788-05bc-4280-a894-5d9a71ef3be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:47.143949 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7d788-05bc-4280-a894-5d9a71ef3be2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:47.144880 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7d788-05bc-4280-a894-5d9a71ef3be2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:47.146493 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7d788-05bc-4280-a894-5d9a71ef3be2 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:47.157571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1133/2242] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:49:47.161745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741c319-1b67-4897-a512-2eeae89b5fb6 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:47.172165 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741c319-1b67-4897-a512-2eeae89b5fb6 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:47.177398 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ecd7c-0482-4966-9327-11ac9ecf86c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:47.178083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ecd7c-0482-4966-9327-11ac9ecf86c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:47.178359 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ecd7c-0482-4966-9327-11ac9ecf86c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:47.178628 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ecd7c-0482-4966-9327-11ac9ecf86c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:47.179354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ecd7c-0482-4966-9327-11ac9ecf86c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:47.181011 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ecd7c-0482-4966-9327-11ac9ecf86c1 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:47.192150 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1110/2243] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:49:47.198071 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7d788-05bc-4280-a894-5d9a71ef3be2 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:47.212800 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7d788-05bc-4280-a894-5d9a71ef3be2 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:47.219988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40990909-d431-499c-b4fc-32660657afa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:47.221368 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40990909-d431-499c-b4fc-32660657afa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:47.221711 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40990909-d431-499c-b4fc-32660657afa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:47.221974 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40990909-d431-499c-b4fc-32660657afa9 admin admin] PATH_INFO: `/v3/projects/08be7238cdf9458385660a09dfef93a7/users/70f51c48e184409982b17a3fa5d35d3e/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:47.237587 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40990909-d431-499c-b4fc-32660657afa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08be7238cdf9458385660a09dfef93a7, user_id=70f51c48e184409982b17a3fa5d35d3e, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:47.240289 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40990909-d431-499c-b4fc-32660657afa9 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:47.256418 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1134/2244] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:49:47 2022] PUT /identity/v3/projects/08be7238cdf9458385660a09dfef93a7/users/70f51c48e184409982b17a3fa5d35d3e/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:49:47.267284 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6561022-e2ae-4d9e-a05d-6891257640c7 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:47.267575 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6561022-e2ae-4d9e-a05d-6891257640c7 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:47.267846 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6561022-e2ae-4d9e-a05d-6891257640c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:47.302754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-a6561022-e2ae-4d9e-a05d-6891257640c7 None None] MFA Rules not processed for user `70f51c48e184409982b17a3fa5d35d3e`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:47.304543 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a6561022-e2ae-4d9e-a05d-6891257640c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:47.384247 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1111/2245] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:47 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:49:47.391342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40990909-d431-499c-b4fc-32660657afa9 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:47.456138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40990909-d431-499c-b4fc-32660657afa9 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:47.461107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979112ac-5983-4584-8cbe-afadbb388d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:47.461816 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979112ac-5983-4584-8cbe-afadbb388d0c admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:47.462095 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979112ac-5983-4584-8cbe-afadbb388d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:47.462369 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979112ac-5983-4584-8cbe-afadbb388d0c admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:47.463110 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979112ac-5983-4584-8cbe-afadbb388d0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:47.465048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979112ac-5983-4584-8cbe-afadbb388d0c admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:47.492782 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1135/2246] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:49:47 2022] POST /identity/v3/projects => generated 367 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:49:47.497845 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6561022-e2ae-4d9e-a05d-6891257640c7 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:47.510114 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6561022-e2ae-4d9e-a05d-6891257640c7 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:47.591170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1de427-ee1d-41f9-ad2b-e16b2d455d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:47.591850 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1de427-ee1d-41f9-ad2b-e16b2d455d17 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:47.592128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1de427-ee1d-41f9-ad2b-e16b2d455d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:47.592405 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1de427-ee1d-41f9-ad2b-e16b2d455d17 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:47.593153 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1de427-ee1d-41f9-ad2b-e16b2d455d17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:47.595138 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1de427-ee1d-41f9-ad2b-e16b2d455d17 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:47.618068 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1112/2247] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:49:47 2022] POST /identity/v3/users => generated 290 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:49:47.622165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979112ac-5983-4584-8cbe-afadbb388d0c admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:47.633375 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979112ac-5983-4584-8cbe-afadbb388d0c admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:47.639440 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2808fc5a-fba8-49f7-8bd2-a74e1c860eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:47.640210 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808fc5a-fba8-49f7-8bd2-a74e1c860eaf admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:47.640584 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808fc5a-fba8-49f7-8bd2-a74e1c860eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:47.641246 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808fc5a-fba8-49f7-8bd2-a74e1c860eaf admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:47.642092 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2808fc5a-fba8-49f7-8bd2-a74e1c860eaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:47.643880 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2808fc5a-fba8-49f7-8bd2-a74e1c860eaf admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:47.655227 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1136/2248] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:49:47.659408 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1de427-ee1d-41f9-ad2b-e16b2d455d17 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:47.669404 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1de427-ee1d-41f9-ad2b-e16b2d455d17 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:47.674316 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51538f-3151-4a7d-af87-2c19e8f4ca79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:47.675203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51538f-3151-4a7d-af87-2c19e8f4ca79 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:47.675447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51538f-3151-4a7d-af87-2c19e8f4ca79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:47.675812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51538f-3151-4a7d-af87-2c19e8f4ca79 admin admin] PATH_INFO: `/v3/projects/19b2d121985947bb9e64b82dffe3bf0b/users/1dfd136701d740cc9d418c49d66fc22a/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:47.695164 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51538f-3151-4a7d-af87-2c19e8f4ca79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b2d121985947bb9e64b82dffe3bf0b, user_id=1dfd136701d740cc9d418c49d66fc22a, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:47.697851 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51538f-3151-4a7d-af87-2c19e8f4ca79 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:47.711430 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1113/2249] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:49:47 2022] PUT /identity/v3/projects/19b2d121985947bb9e64b82dffe3bf0b/users/1dfd136701d740cc9d418c49d66fc22a/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:49:47.715933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2808fc5a-fba8-49f7-8bd2-a74e1c860eaf admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:47.725976 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2808fc5a-fba8-49f7-8bd2-a74e1c860eaf admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:47.787010 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf7126-27bb-4495-be0b-13222f2d64f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:47.788219 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf7126-27bb-4495-be0b-13222f2d64f9 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:47.788639 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf7126-27bb-4495-be0b-13222f2d64f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:47.788959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf7126-27bb-4495-be0b-13222f2d64f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:47.790046 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf7126-27bb-4495-be0b-13222f2d64f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:47.792388 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf7126-27bb-4495-be0b-13222f2d64f9 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:47.806069 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1137/2250] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:49:47.810457 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51538f-3151-4a7d-af87-2c19e8f4ca79 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:47.821393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51538f-3151-4a7d-af87-2c19e8f4ca79 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:47.827372 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2b477-d595-4782-98a5-c09f712b2ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:47.828261 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2b477-d595-4782-98a5-c09f712b2ab6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:47.828535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2b477-d595-4782-98a5-c09f712b2ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:47.828846 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2b477-d595-4782-98a5-c09f712b2ab6 admin admin] PATH_INFO: `/v3/projects/19b2d121985947bb9e64b82dffe3bf0b/users/1dfd136701d740cc9d418c49d66fc22a/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:47.831760 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2b477-d595-4782-98a5-c09f712b2ab6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b2d121985947bb9e64b82dffe3bf0b, user_id=1dfd136701d740cc9d418c49d66fc22a, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:47.834657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2b477-d595-4782-98a5-c09f712b2ab6 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:47.854600 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1114/2251] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:49:47 2022] PUT /identity/v3/projects/19b2d121985947bb9e64b82dffe3bf0b/users/1dfd136701d740cc9d418c49d66fc22a/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:49:47.866796 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e5cdd-9681-4abb-9161-3db748879a07 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:47.867081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e5cdd-9681-4abb-9161-3db748879a07 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:47.867352 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e5cdd-9681-4abb-9161-3db748879a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:47.896964 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-0d1e5cdd-9681-4abb-9161-3db748879a07 None None] MFA Rules not processed for user `1dfd136701d740cc9d418c49d66fc22a`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:49:47.898886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-0d1e5cdd-9681-4abb-9161-3db748879a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:47.972955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1138/2252] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:49:47 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:49:47.985973 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2b477-d595-4782-98a5-c09f712b2ab6 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:47.989150 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-eef2b477-d595-4782-98a5-c09f712b2ab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:48.054891 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2b477-d595-4782-98a5-c09f712b2ab6 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:48.058995 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75817e-1f76-4a8c-82fa-9027e3b90738 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:48.059613 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75817e-1f76-4a8c-82fa-9027e3b90738 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:48.059890 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75817e-1f76-4a8c-82fa-9027e3b90738 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:48.060160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75817e-1f76-4a8c-82fa-9027e3b90738 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:48.069487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75817e-1f76-4a8c-82fa-9027e3b90738 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:48.070935 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75817e-1f76-4a8c-82fa-9027e3b90738 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:48.080085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1115/2253] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:49:47 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:49:48.164520 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e5cdd-9681-4abb-9161-3db748879a07 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:49:48.166055 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-0d1e5cdd-9681-4abb-9161-3db748879a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:49:48.178069 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e5cdd-9681-4abb-9161-3db748879a07 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:49:48.181875 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364434b1-003d-48a0-a05f-98d6b107df11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:49:48.182520 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364434b1-003d-48a0-a05f-98d6b107df11 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:49:48.182799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364434b1-003d-48a0-a05f-98d6b107df11 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:49:48.183154 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364434b1-003d-48a0-a05f-98d6b107df11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:49:48.240498 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364434b1-003d-48a0-a05f-98d6b107df11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:49:48.241915 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364434b1-003d-48a0-a05f-98d6b107df11 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:49:48.250598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1139/2254] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:49:48 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:50:01.433295 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75817e-1f76-4a8c-82fa-9027e3b90738 service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:01.498980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75817e-1f76-4a8c-82fa-9027e3b90738 service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:01.504177 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a665aadc-fb09-406a-8b7f-21e413bf2754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:01.505104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665aadc-fb09-406a-8b7f-21e413bf2754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:01.505385 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665aadc-fb09-406a-8b7f-21e413bf2754 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:01.505650 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665aadc-fb09-406a-8b7f-21e413bf2754 admin admin] PATH_INFO: `/v3/users/70f51c48e184409982b17a3fa5d35d3e` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:01.506926 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665aadc-fb09-406a-8b7f-21e413bf2754 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70f51c48e184409982b17a3fa5d35d3e)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:01.508894 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665aadc-fb09-406a-8b7f-21e413bf2754 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:01.561213 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-a665aadc-fb09-406a-8b7f-21e413bf2754 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70f51c48e184409982b17a3fa5d35d3e'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:01.568277 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-a665aadc-fb09-406a-8b7f-21e413bf2754 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70f51c48e184409982b17a3fa5d35d3e'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:01.575228 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-a665aadc-fb09-406a-8b7f-21e413bf2754 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70f51c48e184409982b17a3fa5d35d3e'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:01.575916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-a665aadc-fb09-406a-8b7f-21e413bf2754 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70f51c48e184409982b17a3fa5d35d3e'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:01.576672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-a665aadc-fb09-406a-8b7f-21e413bf2754 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70f51c48e184409982b17a3fa5d35d3e'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:01.593880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1116/2255] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:50:01 2022] DELETE /identity/v3/users/70f51c48e184409982b17a3fa5d35d3e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:01.598196 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd005c-2e11-47ea-ab55-a766bd01a251 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:01.598697 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd005c-2e11-47ea-ab55-a766bd01a251 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:01.598830 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd005c-2e11-47ea-ab55-a766bd01a251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:01.618211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-91dd005c-2e11-47ea-ab55-a766bd01a251 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:01.619649 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-91dd005c-2e11-47ea-ab55-a766bd01a251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:01.695330 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1140/2256] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:01.704174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a665aadc-fb09-406a-8b7f-21e413bf2754 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:01.705573 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a665aadc-fb09-406a-8b7f-21e413bf2754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:01.761741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a665aadc-fb09-406a-8b7f-21e413bf2754 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:01.765347 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259201ca-3eaa-494f-91d4-8b5d0d480604 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:01.765999 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259201ca-3eaa-494f-91d4-8b5d0d480604 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:01.766316 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259201ca-3eaa-494f-91d4-8b5d0d480604 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:01.766583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259201ca-3eaa-494f-91d4-8b5d0d480604 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:01.775723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259201ca-3eaa-494f-91d4-8b5d0d480604 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:01.777207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259201ca-3eaa-494f-91d4-8b5d0d480604 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:01.786376 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1117/2257] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:50:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:50:02.065677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd005c-2e11-47ea-ab55-a766bd01a251 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:02.067283 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-91dd005c-2e11-47ea-ab55-a766bd01a251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:02.079509 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd005c-2e11-47ea-ab55-a766bd01a251 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:02.084982 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804d6f9-bd85-4668-8a5c-90dd07f96cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:02.085867 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804d6f9-bd85-4668-8a5c-90dd07f96cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:02.086128 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804d6f9-bd85-4668-8a5c-90dd07f96cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:02.086398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804d6f9-bd85-4668-8a5c-90dd07f96cd7 admin admin] PATH_INFO: `/v3/projects/08be7238cdf9458385660a09dfef93a7` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:02.087682 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804d6f9-bd85-4668-8a5c-90dd07f96cd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08be7238cdf9458385660a09dfef93a7)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:02.089657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804d6f9-bd85-4668-8a5c-90dd07f96cd7 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:02.138524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-4804d6f9-bd85-4668-8a5c-90dd07f96cd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08be7238cdf9458385660a09dfef93a7', 'request_id': 'req-4804d6f9-bd85-4668-8a5c-90dd07f96cd7'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:02.144739 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-4804d6f9-bd85-4668-8a5c-90dd07f96cd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08be7238cdf9458385660a09dfef93a7', 'request_id': 'req-4804d6f9-bd85-4668-8a5c-90dd07f96cd7'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:02.150072 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-4804d6f9-bd85-4668-8a5c-90dd07f96cd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08be7238cdf9458385660a09dfef93a7', 'request_id': 'req-4804d6f9-bd85-4668-8a5c-90dd07f96cd7'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:02.162896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-4804d6f9-bd85-4668-8a5c-90dd07f96cd7 admin admin] The token cache is being invalidate because project 08be7238cdf9458385660a09dfef93a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:50:02.163190 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-4804d6f9-bd85-4668-8a5c-90dd07f96cd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:02.163795 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-4804d6f9-bd85-4668-8a5c-90dd07f96cd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:02.165420 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1141/2258] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:50:02 2022] DELETE /identity/v3/projects/08be7238cdf9458385660a09dfef93a7 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:02.169475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259201ca-3eaa-494f-91d4-8b5d0d480604 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:02.179198 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259201ca-3eaa-494f-91d4-8b5d0d480604 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:02.236882 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d30ca-2daf-4cbc-9b85-4558ffcc851c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:02.237759 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d30ca-2daf-4cbc-9b85-4558ffcc851c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:02.238038 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d30ca-2daf-4cbc-9b85-4558ffcc851c admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:02.238314 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d30ca-2daf-4cbc-9b85-4558ffcc851c admin admin] PATH_INFO: `/v3/users/1dfd136701d740cc9d418c49d66fc22a` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:02.239714 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d30ca-2daf-4cbc-9b85-4558ffcc851c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dfd136701d740cc9d418c49d66fc22a)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:02.243005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d30ca-2daf-4cbc-9b85-4558ffcc851c admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:02.301012 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-744d30ca-2daf-4cbc-9b85-4558ffcc851c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dfd136701d740cc9d418c49d66fc22a'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:02.307685 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-744d30ca-2daf-4cbc-9b85-4558ffcc851c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dfd136701d740cc9d418c49d66fc22a'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:02.315079 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-744d30ca-2daf-4cbc-9b85-4558ffcc851c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dfd136701d740cc9d418c49d66fc22a'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:02.315925 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-744d30ca-2daf-4cbc-9b85-4558ffcc851c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dfd136701d740cc9d418c49d66fc22a'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:02.316690 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-744d30ca-2daf-4cbc-9b85-4558ffcc851c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dfd136701d740cc9d418c49d66fc22a'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:02.333637 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1118/2259] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:50:02 2022] DELETE /identity/v3/users/1dfd136701d740cc9d418c49d66fc22a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:02.622338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804d6f9-bd85-4668-8a5c-90dd07f96cd7 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:02.623851 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-4804d6f9-bd85-4668-8a5c-90dd07f96cd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:02.639995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804d6f9-bd85-4668-8a5c-90dd07f96cd7 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:02.697792 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3113b-2173-4c76-85f9-4e9046b6009a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:02.698653 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3113b-2173-4c76-85f9-4e9046b6009a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:02.698931 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3113b-2173-4c76-85f9-4e9046b6009a admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:02.699199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3113b-2173-4c76-85f9-4e9046b6009a admin admin] PATH_INFO: `/v3/projects/19b2d121985947bb9e64b82dffe3bf0b` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:02.700378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f3113b-2173-4c76-85f9-4e9046b6009a admin admin] RBAC: Authorizing `identity:delete_project(project_id=19b2d121985947bb9e64b82dffe3bf0b)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:02.702336 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f3113b-2173-4c76-85f9-4e9046b6009a admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:02.752021 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-49f3113b-2173-4c76-85f9-4e9046b6009a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19b2d121985947bb9e64b82dffe3bf0b', 'request_id': 'req-49f3113b-2173-4c76-85f9-4e9046b6009a'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:02.758165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-49f3113b-2173-4c76-85f9-4e9046b6009a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19b2d121985947bb9e64b82dffe3bf0b', 'request_id': 'req-49f3113b-2173-4c76-85f9-4e9046b6009a'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:02.763460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-49f3113b-2173-4c76-85f9-4e9046b6009a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19b2d121985947bb9e64b82dffe3bf0b', 'request_id': 'req-49f3113b-2173-4c76-85f9-4e9046b6009a'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:02.778444 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-49f3113b-2173-4c76-85f9-4e9046b6009a admin admin] The token cache is being invalidate because project 19b2d121985947bb9e64b82dffe3bf0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:50:02.778756 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-49f3113b-2173-4c76-85f9-4e9046b6009a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:02.779537 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-49f3113b-2173-4c76-85f9-4e9046b6009a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:02.781624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1142/2260] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:50:02 2022] DELETE /identity/v3/projects/19b2d121985947bb9e64b82dffe3bf0b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:02.786563 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e43c89-746b-47e8-9f22-4bdc1da9cc5e None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:02.787069 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e43c89-746b-47e8-9f22-4bdc1da9cc5e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:02.788242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e43c89-746b-47e8-9f22-4bdc1da9cc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:02.811633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-15e43c89-746b-47e8-9f22-4bdc1da9cc5e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:02.813029 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-15e43c89-746b-47e8-9f22-4bdc1da9cc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:02.878568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1119/2261] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:02.886453 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65802c-3bd7-48e1-99ba-6158a7cb9413 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:02.886743 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65802c-3bd7-48e1-99ba-6158a7cb9413 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:02.887020 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65802c-3bd7-48e1-99ba-6158a7cb9413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:02.910620 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-0e65802c-3bd7-48e1-99ba-6158a7cb9413 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:02.912279 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-0e65802c-3bd7-48e1-99ba-6158a7cb9413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:02.922365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1143/2262] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:02.927940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e43c89-746b-47e8-9f22-4bdc1da9cc5e None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:02.945725 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e43c89-746b-47e8-9f22-4bdc1da9cc5e None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:02.958758 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c75de-7476-4a61-aadd-0fe944eccc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:02.959467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c75de-7476-4a61-aadd-0fe944eccc26 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:02.959745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c75de-7476-4a61-aadd-0fe944eccc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:02.960063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c75de-7476-4a61-aadd-0fe944eccc26 admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:02.960857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c75de-7476-4a61-aadd-0fe944eccc26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:50:02.961172 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c75de-7476-4a61-aadd-0fe944eccc26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:02.962855 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c75de-7476-4a61-aadd-0fe944eccc26 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:02.977157 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1120/2263] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:50:02.981514 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65802c-3bd7-48e1-99ba-6158a7cb9413 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:02.992754 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65802c-3bd7-48e1-99ba-6158a7cb9413 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:02.999009 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f409779-7692-46fd-8191-2d414635c03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:02.999910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f409779-7692-46fd-8191-2d414635c03d admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:03.000475 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f409779-7692-46fd-8191-2d414635c03d admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:03.000933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f409779-7692-46fd-8191-2d414635c03d admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:03.001962 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f409779-7692-46fd-8191-2d414635c03d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:03.004055 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f409779-7692-46fd-8191-2d414635c03d admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:03.033972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1144/2264] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:50:02 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:50:03.038231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c75de-7476-4a61-aadd-0fe944eccc26 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:03.047949 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c75de-7476-4a61-aadd-0fe944eccc26 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:03.102188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363497f-b711-40ed-beba-506367299d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:03.102853 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363497f-b711-40ed-beba-506367299d9c admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:03.103133 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363497f-b711-40ed-beba-506367299d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:03.103418 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363497f-b711-40ed-beba-506367299d9c admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:03.104122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363497f-b711-40ed-beba-506367299d9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:03.106086 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363497f-b711-40ed-beba-506367299d9c admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:03.129623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1121/2265] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:50:03 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:50:03.134174 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f409779-7692-46fd-8191-2d414635c03d admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:03.144381 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f409779-7692-46fd-8191-2d414635c03d admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:03.151188 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68def27d-32b2-4b8c-a5ee-83fa039135d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:03.151885 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68def27d-32b2-4b8c-a5ee-83fa039135d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:03.152159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68def27d-32b2-4b8c-a5ee-83fa039135d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:03.152427 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68def27d-32b2-4b8c-a5ee-83fa039135d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:03.153204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68def27d-32b2-4b8c-a5ee-83fa039135d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:03.154825 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68def27d-32b2-4b8c-a5ee-83fa039135d7 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:03.165266 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1145/2266] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:50:03.170979 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363497f-b711-40ed-beba-506367299d9c admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:03.186444 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363497f-b711-40ed-beba-506367299d9c admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:03.193769 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ba6d2-a346-483f-a1c1-d357e39f4ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:03.194422 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ba6d2-a346-483f-a1c1-d357e39f4ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:03.194702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ba6d2-a346-483f-a1c1-d357e39f4ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:03.194971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ba6d2-a346-483f-a1c1-d357e39f4ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:03.195695 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ba6d2-a346-483f-a1c1-d357e39f4ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:03.197358 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ba6d2-a346-483f-a1c1-d357e39f4ec8 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:03.207118 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1122/2267] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:50:03.211108 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68def27d-32b2-4b8c-a5ee-83fa039135d7 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:03.221395 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68def27d-32b2-4b8c-a5ee-83fa039135d7 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:03.227367 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e1efb-b9f2-4027-8d46-94e565fad19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:03.228205 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e1efb-b9f2-4027-8d46-94e565fad19e admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:03.228487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e1efb-b9f2-4027-8d46-94e565fad19e admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:03.228774 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e1efb-b9f2-4027-8d46-94e565fad19e admin admin] PATH_INFO: `/v3/projects/6caf756bc14a4d0c9934b1a7bd969121/users/948801c76c88442dab13ddcb02a047d4/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:03.244613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e1efb-b9f2-4027-8d46-94e565fad19e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6caf756bc14a4d0c9934b1a7bd969121, user_id=948801c76c88442dab13ddcb02a047d4, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:03.247321 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e1efb-b9f2-4027-8d46-94e565fad19e admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:03.261235 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1146/2268] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:50:03 2022] PUT /identity/v3/projects/6caf756bc14a4d0c9934b1a7bd969121/users/948801c76c88442dab13ddcb02a047d4/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:50:03.271768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7133d-3262-4a2d-9583-7c4a5c52ddd3 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:03.272050 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7133d-3262-4a2d-9583-7c4a5c52ddd3 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:03.272316 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7133d-3262-4a2d-9583-7c4a5c52ddd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:03.298271 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-edc7133d-3262-4a2d-9583-7c4a5c52ddd3 None None] MFA Rules not processed for user `948801c76c88442dab13ddcb02a047d4`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:03.299451 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-edc7133d-3262-4a2d-9583-7c4a5c52ddd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:03.371541 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1123/2269] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:03.376534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e1efb-b9f2-4027-8d46-94e565fad19e admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:03.443689 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e1efb-b9f2-4027-8d46-94e565fad19e admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:03.449473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260abc1-2a4b-4e74-a1d2-7ac8b3c10651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:03.450122 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260abc1-2a4b-4e74-a1d2-7ac8b3c10651 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:03.450393 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260abc1-2a4b-4e74-a1d2-7ac8b3c10651 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:03.450660 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260abc1-2a4b-4e74-a1d2-7ac8b3c10651 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:03.451369 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4260abc1-2a4b-4e74-a1d2-7ac8b3c10651 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:03.453352 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4260abc1-2a4b-4e74-a1d2-7ac8b3c10651 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:03.483013 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1147/2270] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:50:03 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:50:03.489208 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7133d-3262-4a2d-9583-7c4a5c52ddd3 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:03.503958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7133d-3262-4a2d-9583-7c4a5c52ddd3 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:03.563090 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28dbe40-d361-4aa0-80d6-c1a0fc5e4820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:03.563790 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28dbe40-d361-4aa0-80d6-c1a0fc5e4820 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:03.564056 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28dbe40-d361-4aa0-80d6-c1a0fc5e4820 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:03.564307 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28dbe40-d361-4aa0-80d6-c1a0fc5e4820 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:03.565026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28dbe40-d361-4aa0-80d6-c1a0fc5e4820 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:03.567008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28dbe40-d361-4aa0-80d6-c1a0fc5e4820 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:03.591275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1124/2271] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:50:03 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:50:03.595332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260abc1-2a4b-4e74-a1d2-7ac8b3c10651 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:03.611322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260abc1-2a4b-4e74-a1d2-7ac8b3c10651 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:03.617954 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b8634-a541-4ed4-ba84-ba8d80190f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:03.618608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b8634-a541-4ed4-ba84-ba8d80190f75 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:03.618889 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b8634-a541-4ed4-ba84-ba8d80190f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:03.619161 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b8634-a541-4ed4-ba84-ba8d80190f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:03.619899 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3b8634-a541-4ed4-ba84-ba8d80190f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:03.621536 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3b8634-a541-4ed4-ba84-ba8d80190f75 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:03.632108 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1148/2272] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:50:03.636059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28dbe40-d361-4aa0-80d6-c1a0fc5e4820 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:03.646536 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28dbe40-d361-4aa0-80d6-c1a0fc5e4820 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:03.651731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b78cc1-0901-4bb4-a36e-6658818f357c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:03.652379 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b78cc1-0901-4bb4-a36e-6658818f357c admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:03.652652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b78cc1-0901-4bb4-a36e-6658818f357c admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:03.652962 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b78cc1-0901-4bb4-a36e-6658818f357c admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:03.653680 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b78cc1-0901-4bb4-a36e-6658818f357c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:03.655285 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b78cc1-0901-4bb4-a36e-6658818f357c admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:03.665314 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1125/2273] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:50:03.669483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b8634-a541-4ed4-ba84-ba8d80190f75 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:03.679714 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b8634-a541-4ed4-ba84-ba8d80190f75 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:03.685188 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41e8d7-b875-4445-8947-6a61604c8a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:03.686091 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41e8d7-b875-4445-8947-6a61604c8a4c admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:03.686371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41e8d7-b875-4445-8947-6a61604c8a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:03.686639 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41e8d7-b875-4445-8947-6a61604c8a4c admin admin] PATH_INFO: `/v3/projects/f45496de904d496fb3ba44f1c9b636c9/users/6c4c8e9cd2764ae3be454da1d460005d/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:03.702088 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41e8d7-b875-4445-8947-6a61604c8a4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f45496de904d496fb3ba44f1c9b636c9, user_id=6c4c8e9cd2764ae3be454da1d460005d, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:03.704708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41e8d7-b875-4445-8947-6a61604c8a4c admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:03.719083 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1149/2274] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:50:03 2022] PUT /identity/v3/projects/f45496de904d496fb3ba44f1c9b636c9/users/6c4c8e9cd2764ae3be454da1d460005d/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:03.730550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a549f-5121-44b4-a61b-ec32d3c062a9 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:03.730856 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a549f-5121-44b4-a61b-ec32d3c062a9 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:03.731120 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a549f-5121-44b4-a61b-ec32d3c062a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:03.760032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-8d0a549f-5121-44b4-a61b-ec32d3c062a9 None None] MFA Rules not processed for user `6c4c8e9cd2764ae3be454da1d460005d`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:03.761270 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-8d0a549f-5121-44b4-a61b-ec32d3c062a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:03.826054 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1126/2275] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:03.830875 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41e8d7-b875-4445-8947-6a61604c8a4c admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:03.896890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41e8d7-b875-4445-8947-6a61604c8a4c admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:03.903042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6eb35-b961-48e2-8bad-8145dff77af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:03.903828 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6eb35-b961-48e2-8bad-8145dff77af8 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:03.904100 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6eb35-b961-48e2-8bad-8145dff77af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:03.904359 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6eb35-b961-48e2-8bad-8145dff77af8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:03.905104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb6eb35-b961-48e2-8bad-8145dff77af8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:03.906949 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb6eb35-b961-48e2-8bad-8145dff77af8 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:03.934518 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1150/2276] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:50:03 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:50:03.938795 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a549f-5121-44b4-a61b-ec32d3c062a9 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:03.949860 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a549f-5121-44b4-a61b-ec32d3c062a9 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:04.015581 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8865880-4180-4ba7-ba3f-aac289ce986e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:04.016357 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8865880-4180-4ba7-ba3f-aac289ce986e admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:04.016728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8865880-4180-4ba7-ba3f-aac289ce986e admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:04.017080 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8865880-4180-4ba7-ba3f-aac289ce986e admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:04.017857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8865880-4180-4ba7-ba3f-aac289ce986e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:04.019867 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8865880-4180-4ba7-ba3f-aac289ce986e admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:04.046130 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1127/2277] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:50:03 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:50:04.049746 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6eb35-b961-48e2-8bad-8145dff77af8 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:04.060222 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6eb35-b961-48e2-8bad-8145dff77af8 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:04.069241 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a849a-6cd5-4769-9488-d0df49a3cd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:04.070012 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a849a-6cd5-4769-9488-d0df49a3cd3a admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:04.070372 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a849a-6cd5-4769-9488-d0df49a3cd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:04.070724 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a849a-6cd5-4769-9488-d0df49a3cd3a admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:04.071546 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a849a-6cd5-4769-9488-d0df49a3cd3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:04.073221 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a849a-6cd5-4769-9488-d0df49a3cd3a admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:04.084463 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1151/2278] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:50:04.088846 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8865880-4180-4ba7-ba3f-aac289ce986e admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:04.101372 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8865880-4180-4ba7-ba3f-aac289ce986e admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:04.107412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b4a87-2b98-4a68-86e9-4f540e2bc998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:04.108697 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b4a87-2b98-4a68-86e9-4f540e2bc998 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:04.108976 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b4a87-2b98-4a68-86e9-4f540e2bc998 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:04.109241 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b4a87-2b98-4a68-86e9-4f540e2bc998 admin admin] PATH_INFO: `/v3/projects/0ae67fbeb02a4aa5be7aa005af4d6597/users/314df3c511354c06a5f69ec0f85b135a/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:04.124806 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b4a87-2b98-4a68-86e9-4f540e2bc998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ae67fbeb02a4aa5be7aa005af4d6597, user_id=314df3c511354c06a5f69ec0f85b135a, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:04.127473 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b4a87-2b98-4a68-86e9-4f540e2bc998 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:04.144218 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1128/2279] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:50:04 2022] PUT /identity/v3/projects/0ae67fbeb02a4aa5be7aa005af4d6597/users/314df3c511354c06a5f69ec0f85b135a/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:04.148312 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a849a-6cd5-4769-9488-d0df49a3cd3a admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:04.158078 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a849a-6cd5-4769-9488-d0df49a3cd3a admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:04.224644 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb73b25-946b-4ad6-82e9-08bccde63582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:04.225246 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb73b25-946b-4ad6-82e9-08bccde63582 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:04.225520 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb73b25-946b-4ad6-82e9-08bccde63582 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:04.225788 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb73b25-946b-4ad6-82e9-08bccde63582 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:04.226520 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb73b25-946b-4ad6-82e9-08bccde63582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:04.228224 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb73b25-946b-4ad6-82e9-08bccde63582 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:04.238596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1152/2280] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:50:04.243656 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b4a87-2b98-4a68-86e9-4f540e2bc998 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:04.254828 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b4a87-2b98-4a68-86e9-4f540e2bc998 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:04.259991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff911a5-7cc9-4496-a46a-8f13a9ff98f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:04.260897 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff911a5-7cc9-4496-a46a-8f13a9ff98f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:04.261168 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff911a5-7cc9-4496-a46a-8f13a9ff98f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:04.261456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff911a5-7cc9-4496-a46a-8f13a9ff98f4 admin admin] PATH_INFO: `/v3/projects/0ae67fbeb02a4aa5be7aa005af4d6597/users/314df3c511354c06a5f69ec0f85b135a/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:04.263627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff911a5-7cc9-4496-a46a-8f13a9ff98f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ae67fbeb02a4aa5be7aa005af4d6597, user_id=314df3c511354c06a5f69ec0f85b135a, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:04.266313 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff911a5-7cc9-4496-a46a-8f13a9ff98f4 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:04.276831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1129/2281] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:50:04 2022] PUT /identity/v3/projects/0ae67fbeb02a4aa5be7aa005af4d6597/users/314df3c511354c06a5f69ec0f85b135a/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:04.287130 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ced669-6e33-43cd-b7b0-b09b621c8ab6 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:04.287419 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ced669-6e33-43cd-b7b0-b09b621c8ab6 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:04.287691 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ced669-6e33-43cd-b7b0-b09b621c8ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:04.316804 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-51ced669-6e33-43cd-b7b0-b09b621c8ab6 None None] MFA Rules not processed for user `314df3c511354c06a5f69ec0f85b135a`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:04.318081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-51ced669-6e33-43cd-b7b0-b09b621c8ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:04.395137 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1153/2282] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:04.407093 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff911a5-7cc9-4496-a46a-8f13a9ff98f4 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:04.409071 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-0ff911a5-7cc9-4496-a46a-8f13a9ff98f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:04.484515 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff911a5-7cc9-4496-a46a-8f13a9ff98f4 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:04.489014 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b6201-d9ee-4143-b83f-ed4a4f399333 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:04.489865 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b6201-d9ee-4143-b83f-ed4a4f399333 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:04.490137 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b6201-d9ee-4143-b83f-ed4a4f399333 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:04.490402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b6201-d9ee-4143-b83f-ed4a4f399333 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:04.552753 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b6201-d9ee-4143-b83f-ed4a4f399333 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:04.554846 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b6201-d9ee-4143-b83f-ed4a4f399333 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:04.566268 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1130/2283] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:50:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:50:08.175817 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ced669-6e33-43cd-b7b0-b09b621c8ab6 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:08.177402 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-51ced669-6e33-43cd-b7b0-b09b621c8ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:08.190206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ced669-6e33-43cd-b7b0-b09b621c8ab6 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:08.194340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05e3e1-a0d6-42b1-ab4f-10150c99e677 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:08.194985 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05e3e1-a0d6-42b1-ab4f-10150c99e677 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:08.195266 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05e3e1-a0d6-42b1-ab4f-10150c99e677 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:08.195536 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05e3e1-a0d6-42b1-ab4f-10150c99e677 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:08.251916 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05e3e1-a0d6-42b1-ab4f-10150c99e677 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:08.253419 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05e3e1-a0d6-42b1-ab4f-10150c99e677 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:08.262748 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1154/2284] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:50:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:50:08.916349 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b6201-d9ee-4143-b83f-ed4a4f399333 service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:08.926064 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b6201-d9ee-4143-b83f-ed4a4f399333 service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:08.937259 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d9904-6fcc-4179-bbc7-f9abf797019b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:08.938089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d9904-6fcc-4179-bbc7-f9abf797019b service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:08.938377 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d9904-6fcc-4179-bbc7-f9abf797019b service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:08.938648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d9904-6fcc-4179-bbc7-f9abf797019b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:08.956810 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d9904-6fcc-4179-bbc7-f9abf797019b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:08.958613 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d9904-6fcc-4179-bbc7-f9abf797019b service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:08.972769 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1131/2285] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:50:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:50:09.122294 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05e3e1-a0d6-42b1-ab4f-10150c99e677 service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:09.123835 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-fa05e3e1-a0d6-42b1-ab4f-10150c99e677 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:09.181572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05e3e1-a0d6-42b1-ab4f-10150c99e677 service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:09.185742 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc8f88-b8bb-4e0b-b301-1951dc7f4b9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:09.186395 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc8f88-b8bb-4e0b-b301-1951dc7f4b9f service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:09.186673 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc8f88-b8bb-4e0b-b301-1951dc7f4b9f service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:09.186976 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc8f88-b8bb-4e0b-b301-1951dc7f4b9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:09.188932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-49dc8f88-b8bb-4e0b-b301-1951dc7f4b9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:09.252357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc8f88-b8bb-4e0b-b301-1951dc7f4b9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:09.254038 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc8f88-b8bb-4e0b-b301-1951dc7f4b9f service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:09.264429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1155/2286] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:50:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:50:10.041353 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d9904-6fcc-4179-bbc7-f9abf797019b service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:10.042894 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-2c7d9904-6fcc-4179-bbc7-f9abf797019b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:10.110645 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d9904-6fcc-4179-bbc7-f9abf797019b service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:10.115842 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827e051-0cfa-4362-8d44-636810bb072a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:10.118038 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827e051-0cfa-4362-8d44-636810bb072a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:10.118038 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827e051-0cfa-4362-8d44-636810bb072a admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:10.118038 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827e051-0cfa-4362-8d44-636810bb072a admin admin] PATH_INFO: `/v3/users/5c56cde8b9464350b83fb6824297b077` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:10.133238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827e051-0cfa-4362-8d44-636810bb072a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c56cde8b9464350b83fb6824297b077)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:10.135203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827e051-0cfa-4362-8d44-636810bb072a admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:10.191273 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-5827e051-0cfa-4362-8d44-636810bb072a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c56cde8b9464350b83fb6824297b077'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:10.198946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-5827e051-0cfa-4362-8d44-636810bb072a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c56cde8b9464350b83fb6824297b077'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:10.210580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-5827e051-0cfa-4362-8d44-636810bb072a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c56cde8b9464350b83fb6824297b077'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:10.211665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-5827e051-0cfa-4362-8d44-636810bb072a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c56cde8b9464350b83fb6824297b077'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:10.212600 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-5827e051-0cfa-4362-8d44-636810bb072a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c56cde8b9464350b83fb6824297b077'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:10.231268 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1132/2287] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:50:10 2022] DELETE /identity/v3/users/5c56cde8b9464350b83fb6824297b077 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:10.235999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf777ce-3105-4887-a305-abb9e9ee3cd9 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:10.237433 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf777ce-3105-4887-a305-abb9e9ee3cd9 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:10.237710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf777ce-3105-4887-a305-abb9e9ee3cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:10.256911 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-faf777ce-3105-4887-a305-abb9e9ee3cd9 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:10.258191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-faf777ce-3105-4887-a305-abb9e9ee3cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:10.328310 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1156/2288] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:50:10.338959 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827e051-0cfa-4362-8d44-636810bb072a admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:10.340859 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-5827e051-0cfa-4362-8d44-636810bb072a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:10.407536 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827e051-0cfa-4362-8d44-636810bb072a admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:10.412518 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397fa1b-7b8b-4bf4-8c26-d6ac52d3bbb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:10.413199 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397fa1b-7b8b-4bf4-8c26-d6ac52d3bbb0 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:10.413577 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397fa1b-7b8b-4bf4-8c26-d6ac52d3bbb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:10.414012 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397fa1b-7b8b-4bf4-8c26-d6ac52d3bbb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:10.437808 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397fa1b-7b8b-4bf4-8c26-d6ac52d3bbb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:10.439306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397fa1b-7b8b-4bf4-8c26-d6ac52d3bbb0 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:10.450800 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1133/2289] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:50:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:50:10.754752 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf777ce-3105-4887-a305-abb9e9ee3cd9 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:10.758056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-faf777ce-3105-4887-a305-abb9e9ee3cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:10.775360 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf777ce-3105-4887-a305-abb9e9ee3cd9 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:10.781780 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a956b1-a301-4025-907b-a641285288b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:10.783105 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a956b1-a301-4025-907b-a641285288b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:10.783742 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a956b1-a301-4025-907b-a641285288b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:10.783864 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a956b1-a301-4025-907b-a641285288b6 admin admin] PATH_INFO: `/v3/projects/9a382679a3a042ecb1fb011fe7c642f8` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:10.785475 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a956b1-a301-4025-907b-a641285288b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a382679a3a042ecb1fb011fe7c642f8)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:10.789993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a956b1-a301-4025-907b-a641285288b6 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:10.841631 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-d7a956b1-a301-4025-907b-a641285288b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a382679a3a042ecb1fb011fe7c642f8', 'request_id': 'req-d7a956b1-a301-4025-907b-a641285288b6'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:10.848846 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-d7a956b1-a301-4025-907b-a641285288b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a382679a3a042ecb1fb011fe7c642f8', 'request_id': 'req-d7a956b1-a301-4025-907b-a641285288b6'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:10.854247 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-d7a956b1-a301-4025-907b-a641285288b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a382679a3a042ecb1fb011fe7c642f8', 'request_id': 'req-d7a956b1-a301-4025-907b-a641285288b6'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:10.869648 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-d7a956b1-a301-4025-907b-a641285288b6 admin admin] The token cache is being invalidate because project 9a382679a3a042ecb1fb011fe7c642f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:50:10.869930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-d7a956b1-a301-4025-907b-a641285288b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:10.870554 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-d7a956b1-a301-4025-907b-a641285288b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:10.872143 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1157/2290] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:50:10 2022] DELETE /identity/v3/projects/9a382679a3a042ecb1fb011fe7c642f8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:10.878376 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeec747-5e19-4170-99f7-d4f6d1285ae7 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:10.879165 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeec747-5e19-4170-99f7-d4f6d1285ae7 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:10.879462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeec747-5e19-4170-99f7-d4f6d1285ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:10.898681 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-cbeec747-5e19-4170-99f7-d4f6d1285ae7 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:10.899924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-cbeec747-5e19-4170-99f7-d4f6d1285ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:10.961755 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1134/2291] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:10.969700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d864b0-e650-4d84-9342-89e8ed32513d None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:10.970013 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d864b0-e650-4d84-9342-89e8ed32513d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:10.970914 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d864b0-e650-4d84-9342-89e8ed32513d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:10.989310 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-f8d864b0-e650-4d84-9342-89e8ed32513d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:10.990678 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-f8d864b0-e650-4d84-9342-89e8ed32513d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:10.998948 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1158/2292] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:50:11.003162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeec747-5e19-4170-99f7-d4f6d1285ae7 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:11.012753 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeec747-5e19-4170-99f7-d4f6d1285ae7 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:11.017484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fbe983-5a4b-400a-aaf0-b8dd3b69b61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:11.018144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbe983-5a4b-400a-aaf0-b8dd3b69b61c admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:11.018420 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbe983-5a4b-400a-aaf0-b8dd3b69b61c admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:11.018690 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbe983-5a4b-400a-aaf0-b8dd3b69b61c admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:11.019420 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fbe983-5a4b-400a-aaf0-b8dd3b69b61c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:50:11.019706 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fbe983-5a4b-400a-aaf0-b8dd3b69b61c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:11.021528 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fbe983-5a4b-400a-aaf0-b8dd3b69b61c admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:11.033156 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1135/2293] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:50:11.042790 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d864b0-e650-4d84-9342-89e8ed32513d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:11.053742 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d864b0-e650-4d84-9342-89e8ed32513d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:11.060017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b96b94-bb3f-4de2-a8cf-580aa15ee583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:11.060728 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b96b94-bb3f-4de2-a8cf-580aa15ee583 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:11.061007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b96b94-bb3f-4de2-a8cf-580aa15ee583 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:11.061279 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b96b94-bb3f-4de2-a8cf-580aa15ee583 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:11.061996 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b96b94-bb3f-4de2-a8cf-580aa15ee583 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:11.063912 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b96b94-bb3f-4de2-a8cf-580aa15ee583 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:11.091089 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1159/2294] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:50:11 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:50:11.095636 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fbe983-5a4b-400a-aaf0-b8dd3b69b61c admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:11.106213 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fbe983-5a4b-400a-aaf0-b8dd3b69b61c admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:11.164039 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b56bb-6229-460b-8321-d1da1bef956e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:11.164743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b56bb-6229-460b-8321-d1da1bef956e admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:11.165018 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b56bb-6229-460b-8321-d1da1bef956e admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:11.165272 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b56bb-6229-460b-8321-d1da1bef956e admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:11.165962 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b56bb-6229-460b-8321-d1da1bef956e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:11.167921 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b56bb-6229-460b-8321-d1da1bef956e admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:11.191416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1136/2295] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:50:11 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:50:11.192710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b96b94-bb3f-4de2-a8cf-580aa15ee583 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:11.203059 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b96b94-bb3f-4de2-a8cf-580aa15ee583 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:11.208071 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1010a87-ca4d-4292-a526-b25b597631b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:11.208791 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1010a87-ca4d-4292-a526-b25b597631b6 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:11.209073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1010a87-ca4d-4292-a526-b25b597631b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:11.209343 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1010a87-ca4d-4292-a526-b25b597631b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:11.210072 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1010a87-ca4d-4292-a526-b25b597631b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:11.211703 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1010a87-ca4d-4292-a526-b25b597631b6 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:11.222389 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1160/2296] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:50:11.226672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b56bb-6229-460b-8321-d1da1bef956e admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:11.237298 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b56bb-6229-460b-8321-d1da1bef956e admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:11.243172 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387855ee-6462-418e-bd8c-186a854eaa17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:11.243851 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387855ee-6462-418e-bd8c-186a854eaa17 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:11.244222 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387855ee-6462-418e-bd8c-186a854eaa17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:11.244624 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387855ee-6462-418e-bd8c-186a854eaa17 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:11.245512 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387855ee-6462-418e-bd8c-186a854eaa17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:11.247135 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387855ee-6462-418e-bd8c-186a854eaa17 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:11.259585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1137/2297] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:50:11.263747 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1010a87-ca4d-4292-a526-b25b597631b6 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:11.275904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1010a87-ca4d-4292-a526-b25b597631b6 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:11.281199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed601e-0c9a-4ed5-94a5-51f59601000f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:11.282132 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed601e-0c9a-4ed5-94a5-51f59601000f admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:11.282409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed601e-0c9a-4ed5-94a5-51f59601000f admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:11.282673 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed601e-0c9a-4ed5-94a5-51f59601000f admin admin] PATH_INFO: `/v3/projects/b1c46939bf6f40f19dac93cf5d494ab1/users/cba3e3a904ca41b78e518699f88e315e/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:11.297449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed601e-0c9a-4ed5-94a5-51f59601000f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1c46939bf6f40f19dac93cf5d494ab1, user_id=cba3e3a904ca41b78e518699f88e315e, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:11.300125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed601e-0c9a-4ed5-94a5-51f59601000f admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:11.313687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1161/2298] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:50:11 2022] PUT /identity/v3/projects/b1c46939bf6f40f19dac93cf5d494ab1/users/cba3e3a904ca41b78e518699f88e315e/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:11.318980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bce321-3be5-4eb6-98cb-95d4afb4a2f2 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:11.319283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bce321-3be5-4eb6-98cb-95d4afb4a2f2 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:11.319510 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bce321-3be5-4eb6-98cb-95d4afb4a2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:11.339521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-32bce321-3be5-4eb6-98cb-95d4afb4a2f2 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:11.340903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-32bce321-3be5-4eb6-98cb-95d4afb4a2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:11.428458 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1138/2299] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:11.439016 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed601e-0c9a-4ed5-94a5-51f59601000f admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:11.441256 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c0ed601e-0c9a-4ed5-94a5-51f59601000f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:11.559771 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed601e-0c9a-4ed5-94a5-51f59601000f admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:11.565085 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f96b3-998b-4386-bc7d-f52ea2686de9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:11.565971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f96b3-998b-4386-bc7d-f52ea2686de9 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:11.569076 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f96b3-998b-4386-bc7d-f52ea2686de9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:11.569511 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f96b3-998b-4386-bc7d-f52ea2686de9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:11.594069 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f96b3-998b-4386-bc7d-f52ea2686de9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:11.595187 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f96b3-998b-4386-bc7d-f52ea2686de9 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:11.613107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1162/2300] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:50:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:50:12.999520 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bce321-3be5-4eb6-98cb-95d4afb4a2f2 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:13.002102 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-32bce321-3be5-4eb6-98cb-95d4afb4a2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:13.026020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bce321-3be5-4eb6-98cb-95d4afb4a2f2 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:13.045800 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba146d-5520-4c25-8fa9-5f66ea7d2fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:13.047153 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba146d-5520-4c25-8fa9-5f66ea7d2fea admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:13.047867 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba146d-5520-4c25-8fa9-5f66ea7d2fea admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:13.048610 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba146d-5520-4c25-8fa9-5f66ea7d2fea admin admin] PATH_INFO: `/v3/users/32a253ad3b764e869aa979d7ba1df6ad` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:13.050018 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ba146d-5520-4c25-8fa9-5f66ea7d2fea admin admin] RBAC: Authorizing `identity:delete_user(user_id=32a253ad3b764e869aa979d7ba1df6ad)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:13.054106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ba146d-5520-4c25-8fa9-5f66ea7d2fea admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:13.130428 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f96b3-998b-4386-bc7d-f52ea2686de9 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:13.132089 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-177f96b3-998b-4386-bc7d-f52ea2686de9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:13.152776 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f96b3-998b-4386-bc7d-f52ea2686de9 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:13.159711 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c4ba146d-5520-4c25-8fa9-5f66ea7d2fea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32a253ad3b764e869aa979d7ba1df6ad'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:13.163541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acaf6c-859e-4e89-b645-544966f09953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:13.164433 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acaf6c-859e-4e89-b645-544966f09953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:13.164952 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acaf6c-859e-4e89-b645-544966f09953 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:13.165222 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acaf6c-859e-4e89-b645-544966f09953 admin admin] PATH_INFO: `/v3/users/948801c76c88442dab13ddcb02a047d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:13.167611 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85acaf6c-859e-4e89-b645-544966f09953 admin admin] RBAC: Authorizing `identity:delete_user(user_id=948801c76c88442dab13ddcb02a047d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:13.170394 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85acaf6c-859e-4e89-b645-544966f09953 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:13.173832 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c4ba146d-5520-4c25-8fa9-5f66ea7d2fea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32a253ad3b764e869aa979d7ba1df6ad'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:13.187793 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c4ba146d-5520-4c25-8fa9-5f66ea7d2fea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32a253ad3b764e869aa979d7ba1df6ad'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:13.188933 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c4ba146d-5520-4c25-8fa9-5f66ea7d2fea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32a253ad3b764e869aa979d7ba1df6ad'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:13.189816 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c4ba146d-5520-4c25-8fa9-5f66ea7d2fea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32a253ad3b764e869aa979d7ba1df6ad'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:13.219581 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1139/2301] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:50:12 2022] DELETE /identity/v3/users/32a253ad3b764e869aa979d7ba1df6ad => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:13.224366 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8de5f9-b9b2-4814-a430-90a6a817b767 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:13.225153 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8de5f9-b9b2-4814-a430-90a6a817b767 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:13.225676 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8de5f9-b9b2-4814-a430-90a6a817b767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:13.243732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-85acaf6c-859e-4e89-b645-544966f09953 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '948801c76c88442dab13ddcb02a047d4'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:13.256289 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-85acaf6c-859e-4e89-b645-544966f09953 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '948801c76c88442dab13ddcb02a047d4'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:13.257273 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-85acaf6c-859e-4e89-b645-544966f09953 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '948801c76c88442dab13ddcb02a047d4'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:13.257510 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-5d8de5f9-b9b2-4814-a430-90a6a817b767 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:13.258733 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-5d8de5f9-b9b2-4814-a430-90a6a817b767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:13.270799 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-85acaf6c-859e-4e89-b645-544966f09953 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '948801c76c88442dab13ddcb02a047d4'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:13.273111 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-85acaf6c-859e-4e89-b645-544966f09953 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '948801c76c88442dab13ddcb02a047d4'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:13.302011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1163/2302] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:50:13 2022] DELETE /identity/v3/users/948801c76c88442dab13ddcb02a047d4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:50:13.307857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec6c10-2eec-4501-8b06-9810f466e063 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:13.309763 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec6c10-2eec-4501-8b06-9810f466e063 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:13.310265 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec6c10-2eec-4501-8b06-9810f466e063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:13.336269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-7bec6c10-2eec-4501-8b06-9810f466e063 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:13.338214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-7bec6c10-2eec-4501-8b06-9810f466e063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:13.340902 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1140/2303] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:50:13.350524 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8de5f9-b9b2-4814-a430-90a6a817b767 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:13.352534 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-5d8de5f9-b9b2-4814-a430-90a6a817b767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:13.424742 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1164/2304] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:13.428361 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec6c10-2eec-4501-8b06-9810f466e063 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:13.432561 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8de5f9-b9b2-4814-a430-90a6a817b767 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:13.492992 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec6c10-2eec-4501-8b06-9810f466e063 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:13.497400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16f3a5-5fa5-4a4b-80c8-fa77dc3ab803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:13.498053 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16f3a5-5fa5-4a4b-80c8-fa77dc3ab803 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:13.498482 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16f3a5-5fa5-4a4b-80c8-fa77dc3ab803 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:13.499144 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16f3a5-5fa5-4a4b-80c8-fa77dc3ab803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:13.508080 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13cc63-5d35-44a2-b846-822c132fdca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:13.509054 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13cc63-5d35-44a2-b846-822c132fdca2 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:13.509498 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13cc63-5d35-44a2-b846-822c132fdca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:13.509939 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13cc63-5d35-44a2-b846-822c132fdca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:13.561500 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16f3a5-5fa5-4a4b-80c8-fa77dc3ab803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:13.563351 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16f3a5-5fa5-4a4b-80c8-fa77dc3ab803 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:13.567000 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13cc63-5d35-44a2-b846-822c132fdca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:13.568616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13cc63-5d35-44a2-b846-822c132fdca2 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:13.575383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1165/2305] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:50:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 09:50:13.582689 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1141/2306] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:50:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 09:50:13.987532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16f3a5-5fa5-4a4b-80c8-fa77dc3ab803 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:13.989190 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-ee16f3a5-5fa5-4a4b-80c8-fa77dc3ab803 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:14.003618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16f3a5-5fa5-4a4b-80c8-fa77dc3ab803 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:14.009545 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9ae28-66fd-46e7-9ecb-63423d5833b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:14.010082 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9ae28-66fd-46e7-9ecb-63423d5833b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:14.010360 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9ae28-66fd-46e7-9ecb-63423d5833b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:14.010626 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9ae28-66fd-46e7-9ecb-63423d5833b8 admin admin] PATH_INFO: `/v3/projects/6caf756bc14a4d0c9934b1a7bd969121` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:14.011848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9ae28-66fd-46e7-9ecb-63423d5833b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6caf756bc14a4d0c9934b1a7bd969121)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:14.013822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9ae28-66fd-46e7-9ecb-63423d5833b8 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:14.072424 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-03e9ae28-66fd-46e7-9ecb-63423d5833b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6caf756bc14a4d0c9934b1a7bd969121', 'request_id': 'req-03e9ae28-66fd-46e7-9ecb-63423d5833b8'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:14.078556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-03e9ae28-66fd-46e7-9ecb-63423d5833b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6caf756bc14a4d0c9934b1a7bd969121', 'request_id': 'req-03e9ae28-66fd-46e7-9ecb-63423d5833b8'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:14.085170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-03e9ae28-66fd-46e7-9ecb-63423d5833b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6caf756bc14a4d0c9934b1a7bd969121', 'request_id': 'req-03e9ae28-66fd-46e7-9ecb-63423d5833b8'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:14.114232 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-03e9ae28-66fd-46e7-9ecb-63423d5833b8 admin admin] The token cache is being invalidate because project 6caf756bc14a4d0c9934b1a7bd969121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:50:14.114704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-03e9ae28-66fd-46e7-9ecb-63423d5833b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:14.115573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-03e9ae28-66fd-46e7-9ecb-63423d5833b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:14.120251 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1166/2307] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:50:13 2022] DELETE /identity/v3/projects/6caf756bc14a4d0c9934b1a7bd969121 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:50:14.121520 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13cc63-5d35-44a2-b846-822c132fdca2 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:14.132692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13cc63-5d35-44a2-b846-822c132fdca2 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:14.184965 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9ae28-66fd-46e7-9ecb-63423d5833b8 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:14.186877 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-03e9ae28-66fd-46e7-9ecb-63423d5833b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:14.269153 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2f539-5f31-4240-87e3-9fbe5e62c511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:14.270625 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2f539-5f31-4240-87e3-9fbe5e62c511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:14.271086 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2f539-5f31-4240-87e3-9fbe5e62c511 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:14.271534 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2f539-5f31-4240-87e3-9fbe5e62c511 admin admin] PATH_INFO: `/v3/users/6c4c8e9cd2764ae3be454da1d460005d` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:14.284529 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e2f539-5f31-4240-87e3-9fbe5e62c511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c4c8e9cd2764ae3be454da1d460005d)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:14.289108 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e2f539-5f31-4240-87e3-9fbe5e62c511 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:14.316601 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9ae28-66fd-46e7-9ecb-63423d5833b8 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:14.328489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab5a68-834b-4a15-8d19-9e16e7268fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:14.330706 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab5a68-834b-4a15-8d19-9e16e7268fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:14.331453 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab5a68-834b-4a15-8d19-9e16e7268fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:14.331979 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab5a68-834b-4a15-8d19-9e16e7268fe9 admin admin] PATH_INFO: `/v3/projects/b2ffa13056434f0a8fcd41b2470671bc` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:14.333852 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab5a68-834b-4a15-8d19-9e16e7268fe9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2ffa13056434f0a8fcd41b2470671bc)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:14.337320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab5a68-834b-4a15-8d19-9e16e7268fe9 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:14.372787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-f2e2f539-5f31-4240-87e3-9fbe5e62c511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c4c8e9cd2764ae3be454da1d460005d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:14.383988 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-f2e2f539-5f31-4240-87e3-9fbe5e62c511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c4c8e9cd2764ae3be454da1d460005d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:14.400341 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-f2e2f539-5f31-4240-87e3-9fbe5e62c511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c4c8e9cd2764ae3be454da1d460005d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:14.402320 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-f2e2f539-5f31-4240-87e3-9fbe5e62c511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c4c8e9cd2764ae3be454da1d460005d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:14.403565 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-f2e2f539-5f31-4240-87e3-9fbe5e62c511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c4c8e9cd2764ae3be454da1d460005d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:14.424656 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-79ab5a68-834b-4a15-8d19-9e16e7268fe9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2ffa13056434f0a8fcd41b2470671bc', 'request_id': 'req-79ab5a68-834b-4a15-8d19-9e16e7268fe9'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:14.439492 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-79ab5a68-834b-4a15-8d19-9e16e7268fe9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2ffa13056434f0a8fcd41b2470671bc', 'request_id': 'req-79ab5a68-834b-4a15-8d19-9e16e7268fe9'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:14.443822 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1142/2308] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:50:14 2022] DELETE /identity/v3/users/6c4c8e9cd2764ae3be454da1d460005d => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:14.446255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-79ab5a68-834b-4a15-8d19-9e16e7268fe9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2ffa13056434f0a8fcd41b2470671bc', 'request_id': 'req-79ab5a68-834b-4a15-8d19-9e16e7268fe9'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:14.462936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-79ab5a68-834b-4a15-8d19-9e16e7268fe9 admin admin] The token cache is being invalidate because project b2ffa13056434f0a8fcd41b2470671bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:50:14.463497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-79ab5a68-834b-4a15-8d19-9e16e7268fe9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:14.464193 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-79ab5a68-834b-4a15-8d19-9e16e7268fe9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:14.465869 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1167/2309] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:50:14 2022] DELETE /identity/v3/projects/b2ffa13056434f0a8fcd41b2470671bc => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:14.471884 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03411b-cde6-47eb-8d9b-a21d1fd86230 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:14.472164 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03411b-cde6-47eb-8d9b-a21d1fd86230 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:14.473074 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03411b-cde6-47eb-8d9b-a21d1fd86230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:14.518766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-be03411b-cde6-47eb-8d9b-a21d1fd86230 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:14.524544 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-be03411b-cde6-47eb-8d9b-a21d1fd86230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:14.704195 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1143/2310] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:14.707289 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed5d57-6876-49fc-b254-ec203667c9e3 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:14.707692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed5d57-6876-49fc-b254-ec203667c9e3 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:14.708293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed5d57-6876-49fc-b254-ec203667c9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:14.747204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-8eed5d57-6876-49fc-b254-ec203667c9e3 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:14.748416 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-8eed5d57-6876-49fc-b254-ec203667c9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:14.763008 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1168/2311] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:14.767982 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03411b-cde6-47eb-8d9b-a21d1fd86230 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:14.781026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03411b-cde6-47eb-8d9b-a21d1fd86230 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:14.811150 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f932c-30ab-4996-8273-d3184c76bdeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:14.814106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f932c-30ab-4996-8273-d3184c76bdeb admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:14.815693 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f932c-30ab-4996-8273-d3184c76bdeb admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:14.820601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f932c-30ab-4996-8273-d3184c76bdeb admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:14.827233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f932c-30ab-4996-8273-d3184c76bdeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:50:14.827659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f932c-30ab-4996-8273-d3184c76bdeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:14.829522 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f932c-30ab-4996-8273-d3184c76bdeb admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:14.861499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1144/2312] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:50:14.865848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed5d57-6876-49fc-b254-ec203667c9e3 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:14.881459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed5d57-6876-49fc-b254-ec203667c9e3 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:14.892722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21fcf90-25a9-4cae-b1b9-35b5e8a136a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:14.893578 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21fcf90-25a9-4cae-b1b9-35b5e8a136a7 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:14.893951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21fcf90-25a9-4cae-b1b9-35b5e8a136a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:14.894321 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21fcf90-25a9-4cae-b1b9-35b5e8a136a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:14.895222 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21fcf90-25a9-4cae-b1b9-35b5e8a136a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:14.897344 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21fcf90-25a9-4cae-b1b9-35b5e8a136a7 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:14.978639 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1169/2313] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:50:14 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:50:14.979612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f932c-30ab-4996-8273-d3184c76bdeb admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:14.996635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f932c-30ab-4996-8273-d3184c76bdeb admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:15.111074 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523b290-b936-48d2-aedc-052fb3f58a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:15.111886 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523b290-b936-48d2-aedc-052fb3f58a71 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:15.112272 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523b290-b936-48d2-aedc-052fb3f58a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:15.113021 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523b290-b936-48d2-aedc-052fb3f58a71 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:15.113860 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3523b290-b936-48d2-aedc-052fb3f58a71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:15.117479 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3523b290-b936-48d2-aedc-052fb3f58a71 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:15.158916 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21fcf90-25a9-4cae-b1b9-35b5e8a136a7 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:15.159788 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1145/2314] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:50:14 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:50:15.171168 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21fcf90-25a9-4cae-b1b9-35b5e8a136a7 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:15.178978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d5635-bd0f-423d-86e5-261ec25510ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:15.179679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d5635-bd0f-423d-86e5-261ec25510ce admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:15.180201 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d5635-bd0f-423d-86e5-261ec25510ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:15.180476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d5635-bd0f-423d-86e5-261ec25510ce admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:15.181268 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5d5635-bd0f-423d-86e5-261ec25510ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:15.183460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5d5635-bd0f-423d-86e5-261ec25510ce admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:15.200873 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1170/2315] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:50:15.203070 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523b290-b936-48d2-aedc-052fb3f58a71 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:15.214608 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523b290-b936-48d2-aedc-052fb3f58a71 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:15.220170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d908de-859c-4c5b-b8b6-bc6de928a7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:15.221651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d908de-859c-4c5b-b8b6-bc6de928a7e6 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:15.222643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d908de-859c-4c5b-b8b6-bc6de928a7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:15.223081 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d908de-859c-4c5b-b8b6-bc6de928a7e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:15.223994 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d908de-859c-4c5b-b8b6-bc6de928a7e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:15.226143 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d908de-859c-4c5b-b8b6-bc6de928a7e6 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:15.244841 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d5635-bd0f-423d-86e5-261ec25510ce admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:15.256158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d5635-bd0f-423d-86e5-261ec25510ce admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:15.256927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1146/2316] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:50:15.262739 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae2849-f10a-40be-8e63-6796c6a9e565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:15.264404 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae2849-f10a-40be-8e63-6796c6a9e565 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:15.264888 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae2849-f10a-40be-8e63-6796c6a9e565 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:15.265313 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae2849-f10a-40be-8e63-6796c6a9e565 admin admin] PATH_INFO: `/v3/projects/084ec4bab56b4c1d823158e34705bbf0/users/17a673de26884004bc852ef76b18aa06/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:15.289834 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae2849-f10a-40be-8e63-6796c6a9e565 admin admin] RBAC: Authorizing `identity:create_grant(project_id=084ec4bab56b4c1d823158e34705bbf0, user_id=17a673de26884004bc852ef76b18aa06, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:15.294387 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae2849-f10a-40be-8e63-6796c6a9e565 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:15.322203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba119e-f945-466f-bc87-9183f72f0f8b None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:15.322479 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba119e-f945-466f-bc87-9183f72f0f8b None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:15.323031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba119e-f945-466f-bc87-9183f72f0f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:15.324603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1171/2317] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:50:15 2022] PUT /identity/v3/projects/084ec4bab56b4c1d823158e34705bbf0/users/17a673de26884004bc852ef76b18aa06/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:15.356117 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-79ba119e-f945-466f-bc87-9183f72f0f8b None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:15.358218 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-79ba119e-f945-466f-bc87-9183f72f0f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:15.429957 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1147/2318] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:15.441064 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae2849-f10a-40be-8e63-6796c6a9e565 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:15.443962 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-4aae2849-f10a-40be-8e63-6796c6a9e565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:15.544519 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba119e-f945-466f-bc87-9183f72f0f8b None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:15.547039 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-79ba119e-f945-466f-bc87-9183f72f0f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:15.549432 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae2849-f10a-40be-8e63-6796c6a9e565 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:15.555048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f9d4c-3cf2-4b8f-931b-4177c17dbe0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:15.556087 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f9d4c-3cf2-4b8f-931b-4177c17dbe0d service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:15.556504 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f9d4c-3cf2-4b8f-931b-4177c17dbe0d service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:15.557010 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f9d4c-3cf2-4b8f-931b-4177c17dbe0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:15.573724 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f9d4c-3cf2-4b8f-931b-4177c17dbe0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:15.576065 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f9d4c-3cf2-4b8f-931b-4177c17dbe0d service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:15.589571 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba119e-f945-466f-bc87-9183f72f0f8b None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:15.598013 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1172/2319] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:50:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:50:15.601319 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4940e97-58cd-4cf7-be29-6b19d8143325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:15.603197 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4940e97-58cd-4cf7-be29-6b19d8143325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:15.603695 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4940e97-58cd-4cf7-be29-6b19d8143325 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:15.605135 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4940e97-58cd-4cf7-be29-6b19d8143325 admin admin] PATH_INFO: `/v3/projects/f45496de904d496fb3ba44f1c9b636c9` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:15.608020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4940e97-58cd-4cf7-be29-6b19d8143325 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f45496de904d496fb3ba44f1c9b636c9)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:15.611229 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4940e97-58cd-4cf7-be29-6b19d8143325 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:15.715704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c4940e97-58cd-4cf7-be29-6b19d8143325 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f45496de904d496fb3ba44f1c9b636c9', 'request_id': 'req-c4940e97-58cd-4cf7-be29-6b19d8143325'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:15.728131 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c4940e97-58cd-4cf7-be29-6b19d8143325 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f45496de904d496fb3ba44f1c9b636c9', 'request_id': 'req-c4940e97-58cd-4cf7-be29-6b19d8143325'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:15.740429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c4940e97-58cd-4cf7-be29-6b19d8143325 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f45496de904d496fb3ba44f1c9b636c9', 'request_id': 'req-c4940e97-58cd-4cf7-be29-6b19d8143325'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:15.766114 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c4940e97-58cd-4cf7-be29-6b19d8143325 admin admin] The token cache is being invalidate because project f45496de904d496fb3ba44f1c9b636c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:50:15.766482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c4940e97-58cd-4cf7-be29-6b19d8143325 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:15.767135 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c4940e97-58cd-4cf7-be29-6b19d8143325 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:15.776529 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1148/2320] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:50:15 2022] DELETE /identity/v3/projects/f45496de904d496fb3ba44f1c9b636c9 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:15.780716 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f9d4c-3cf2-4b8f-931b-4177c17dbe0d service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:15.795483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f9d4c-3cf2-4b8f-931b-4177c17dbe0d service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:15.947329 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d3588-84c7-4fd1-b9d5-269e579a9586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:15.948256 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d3588-84c7-4fd1-b9d5-269e579a9586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:15.948676 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d3588-84c7-4fd1-b9d5-269e579a9586 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:15.949143 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d3588-84c7-4fd1-b9d5-269e579a9586 admin admin] PATH_INFO: `/v3/users/314df3c511354c06a5f69ec0f85b135a` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:15.961958 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d3588-84c7-4fd1-b9d5-269e579a9586 admin admin] RBAC: Authorizing `identity:delete_user(user_id=314df3c511354c06a5f69ec0f85b135a)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:15.964427 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d3588-84c7-4fd1-b9d5-269e579a9586 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:16.033714 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-157d3588-84c7-4fd1-b9d5-269e579a9586 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '314df3c511354c06a5f69ec0f85b135a'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:16.045842 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-157d3588-84c7-4fd1-b9d5-269e579a9586 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '314df3c511354c06a5f69ec0f85b135a'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:16.046902 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-157d3588-84c7-4fd1-b9d5-269e579a9586 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '314df3c511354c06a5f69ec0f85b135a'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:16.059698 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-157d3588-84c7-4fd1-b9d5-269e579a9586 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '314df3c511354c06a5f69ec0f85b135a'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:16.060732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-157d3588-84c7-4fd1-b9d5-269e579a9586 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '314df3c511354c06a5f69ec0f85b135a'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:16.094622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1173/2321] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:50:15 2022] DELETE /identity/v3/users/314df3c511354c06a5f69ec0f85b135a => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:16.826761 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81925794-9d10-4ddb-b8cf-a958f1c114c6 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:16.827184 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81925794-9d10-4ddb-b8cf-a958f1c114c6 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:16.827280 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81925794-9d10-4ddb-b8cf-a958f1c114c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:16.838037 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d3588-84c7-4fd1-b9d5-269e579a9586 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:16.839728 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-157d3588-84c7-4fd1-b9d5-269e579a9586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:16.874207 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-81925794-9d10-4ddb-b8cf-a958f1c114c6 None None] MFA Rules not processed for user `e58c14dca7da4e6495f7e08f2bb127e2`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:16.875233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-81925794-9d10-4ddb-b8cf-a958f1c114c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:16.942822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d3588-84c7-4fd1-b9d5-269e579a9586 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:16.954443 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ba456-df4b-4c90-a646-c11d3efd3967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:16.955365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ba456-df4b-4c90-a646-c11d3efd3967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:16.956202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ba456-df4b-4c90-a646-c11d3efd3967 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:16.956202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ba456-df4b-4c90-a646-c11d3efd3967 admin admin] PATH_INFO: `/v3/projects/0ae67fbeb02a4aa5be7aa005af4d6597` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:16.957932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ba456-df4b-4c90-a646-c11d3efd3967 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ae67fbeb02a4aa5be7aa005af4d6597)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:16.959981 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ba456-df4b-4c90-a646-c11d3efd3967 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:16.982564 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1149/2322] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:17.031367 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-6b8ba456-df4b-4c90-a646-c11d3efd3967 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ae67fbeb02a4aa5be7aa005af4d6597', 'request_id': 'req-6b8ba456-df4b-4c90-a646-c11d3efd3967'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:17.042424 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-6b8ba456-df4b-4c90-a646-c11d3efd3967 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ae67fbeb02a4aa5be7aa005af4d6597', 'request_id': 'req-6b8ba456-df4b-4c90-a646-c11d3efd3967'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:17.060420 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-6b8ba456-df4b-4c90-a646-c11d3efd3967 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ae67fbeb02a4aa5be7aa005af4d6597', 'request_id': 'req-6b8ba456-df4b-4c90-a646-c11d3efd3967'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:17.079998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-6b8ba456-df4b-4c90-a646-c11d3efd3967 admin admin] The token cache is being invalidate because project 0ae67fbeb02a4aa5be7aa005af4d6597 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:50:17.080283 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-6b8ba456-df4b-4c90-a646-c11d3efd3967 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:17.080987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-6b8ba456-df4b-4c90-a646-c11d3efd3967 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:17.087399 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1174/2323] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:50:16 2022] DELETE /identity/v3/projects/0ae67fbeb02a4aa5be7aa005af4d6597 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 09:50:17.089616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a09fc-c5b1-4e6c-a25a-abf679e6b85e None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:17.090144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a09fc-c5b1-4e6c-a25a-abf679e6b85e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:17.090519 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a09fc-c5b1-4e6c-a25a-abf679e6b85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:17.119015 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b49ff-c96c-4285-a865-5a3a9e3da5b9 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:17.119286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b49ff-c96c-4285-a865-5a3a9e3da5b9 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:17.120954 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b49ff-c96c-4285-a865-5a3a9e3da5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:17.143139 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-533a09fc-c5b1-4e6c-a25a-abf679e6b85e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:17.148524 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-533a09fc-c5b1-4e6c-a25a-abf679e6b85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:17.171401 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-674b49ff-c96c-4285-a865-5a3a9e3da5b9 None None] MFA Rules not processed for user `17a673de26884004bc852ef76b18aa06`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:17.172656 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-674b49ff-c96c-4285-a865-5a3a9e3da5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:17.245132 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1150/2324] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:17.248457 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad640a0-9550-4864-bae0-ca189d210ef5 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:17.248775 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad640a0-9550-4864-bae0-ca189d210ef5 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:17.249082 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad640a0-9550-4864-bae0-ca189d210ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:17.290580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-8ad640a0-9550-4864-bae0-ca189d210ef5 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:17.294787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-8ad640a0-9550-4864-bae0-ca189d210ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:17.323521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1151/2325] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:17.324317 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad640a0-9550-4864-bae0-ca189d210ef5 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:17.334430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1175/2326] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:50:17.342871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad640a0-9550-4864-bae0-ca189d210ef5 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:17.346468 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b49ff-c96c-4285-a865-5a3a9e3da5b9 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:17.347899 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-674b49ff-c96c-4285-a865-5a3a9e3da5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:17.355215 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7985b-bba2-4d14-aafd-6087bb56b7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:17.357655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7985b-bba2-4d14-aafd-6087bb56b7de admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:17.358176 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7985b-bba2-4d14-aafd-6087bb56b7de admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:17.358812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7985b-bba2-4d14-aafd-6087bb56b7de admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:17.360189 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7985b-bba2-4d14-aafd-6087bb56b7de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:50:17.361059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7985b-bba2-4d14-aafd-6087bb56b7de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:17.370464 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7985b-bba2-4d14-aafd-6087bb56b7de admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:17.386018 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1152/2327] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 09:50:17.390260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7985b-bba2-4d14-aafd-6087bb56b7de admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:17.401892 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7985b-bba2-4d14-aafd-6087bb56b7de admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:17.413564 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e714f-2ce1-47f2-8bd3-4f9106f52730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:17.414172 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e714f-2ce1-47f2-8bd3-4f9106f52730 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:17.415541 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e714f-2ce1-47f2-8bd3-4f9106f52730 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:17.416164 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e714f-2ce1-47f2-8bd3-4f9106f52730 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:17.417400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e714f-2ce1-47f2-8bd3-4f9106f52730 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:17.426950 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b49ff-c96c-4285-a865-5a3a9e3da5b9 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:17.433513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1364238-24fe-4e9f-9048-777930e4d429 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:17.438014 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e714f-2ce1-47f2-8bd3-4f9106f52730 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:17.446806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1364238-24fe-4e9f-9048-777930e4d429 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:17.447514 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1364238-24fe-4e9f-9048-777930e4d429 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:17.448022 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1364238-24fe-4e9f-9048-777930e4d429 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:17.459903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1364238-24fe-4e9f-9048-777930e4d429 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:17.461863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1364238-24fe-4e9f-9048-777930e4d429 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:17.492531 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1176/2328] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:50:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:50:17.532469 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1153/2329] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:50:17 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:50:17.533590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1364238-24fe-4e9f-9048-777930e4d429 service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:17.560636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1364238-24fe-4e9f-9048-777930e4d429 service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:17.705515 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48060e49-ac28-4415-9be0-50bcbbbf8f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:17.708381 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48060e49-ac28-4415-9be0-50bcbbbf8f70 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:17.708940 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48060e49-ac28-4415-9be0-50bcbbbf8f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:17.710055 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48060e49-ac28-4415-9be0-50bcbbbf8f70 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:17.711762 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48060e49-ac28-4415-9be0-50bcbbbf8f70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:17.718025 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48060e49-ac28-4415-9be0-50bcbbbf8f70 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:17.771415 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1177/2330] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:50:17 2022] POST /identity/v3/users => generated 292 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:50:17.775766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e714f-2ce1-47f2-8bd3-4f9106f52730 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:17.792510 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e714f-2ce1-47f2-8bd3-4f9106f52730 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:17.801798 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723f4a8-98a3-4873-a494-71cc258ee1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:17.804470 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723f4a8-98a3-4873-a494-71cc258ee1fb admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:17.805910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723f4a8-98a3-4873-a494-71cc258ee1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:17.806208 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723f4a8-98a3-4873-a494-71cc258ee1fb admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:17.809414 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723f4a8-98a3-4873-a494-71cc258ee1fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:17.811541 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723f4a8-98a3-4873-a494-71cc258ee1fb admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:17.833925 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce642e2-fd4d-40aa-b3dc-52988e3e5c86 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:17.835438 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce642e2-fd4d-40aa-b3dc-52988e3e5c86 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:17.835829 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce642e2-fd4d-40aa-b3dc-52988e3e5c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:17.853596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1154/2331] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:50:17.866392 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723f4a8-98a3-4873-a494-71cc258ee1fb admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:17.879421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723f4a8-98a3-4873-a494-71cc258ee1fb admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:17.885286 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff0da4-d0f4-4c11-9031-31f95d5b9995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:17.886193 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff0da4-d0f4-4c11-9031-31f95d5b9995 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:17.886572 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff0da4-d0f4-4c11-9031-31f95d5b9995 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:17.886907 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff0da4-d0f4-4c11-9031-31f95d5b9995 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:17.887710 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff0da4-d0f4-4c11-9031-31f95d5b9995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:17.888455 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-8ce642e2-fd4d-40aa-b3dc-52988e3e5c86 None None] MFA Rules not processed for user `cba3e3a904ca41b78e518699f88e315e`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:17.889895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-8ce642e2-fd4d-40aa-b3dc-52988e3e5c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:17.890432 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff0da4-d0f4-4c11-9031-31f95d5b9995 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:17.907943 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1155/2332] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:50:17.912458 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff0da4-d0f4-4c11-9031-31f95d5b9995 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:17.924060 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff0da4-d0f4-4c11-9031-31f95d5b9995 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:17.929577 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530b5b7-aec2-4fc7-ab0a-62365644b030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:17.930644 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530b5b7-aec2-4fc7-ab0a-62365644b030 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:17.930991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530b5b7-aec2-4fc7-ab0a-62365644b030 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:17.931330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530b5b7-aec2-4fc7-ab0a-62365644b030 admin admin] PATH_INFO: `/v3/projects/5a7d53bd35904af49f40686d29a270be/users/f57ea01b72cb4cbfb6c19959ffd2f318/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:17.949511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530b5b7-aec2-4fc7-ab0a-62365644b030 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a7d53bd35904af49f40686d29a270be, user_id=f57ea01b72cb4cbfb6c19959ffd2f318, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:17.953509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530b5b7-aec2-4fc7-ab0a-62365644b030 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:17.975462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1156/2333] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:50:17 2022] PUT /identity/v3/projects/5a7d53bd35904af49f40686d29a270be/users/f57ea01b72cb4cbfb6c19959ffd2f318/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:17.985985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6c6f4-7b22-449b-826a-60b5140498a3 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:17.986266 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6c6f4-7b22-449b-826a-60b5140498a3 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:17.986530 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6c6f4-7b22-449b-826a-60b5140498a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:18.031687 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-43d6c6f4-7b22-449b-826a-60b5140498a3 None None] MFA Rules not processed for user `f57ea01b72cb4cbfb6c19959ffd2f318`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:18.033429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-43d6c6f4-7b22-449b-826a-60b5140498a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:18.058365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1178/2334] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:50:18.063039 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce642e2-fd4d-40aa-b3dc-52988e3e5c86 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:18.066669 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-8ce642e2-fd4d-40aa-b3dc-52988e3e5c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:18.103212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce642e2-fd4d-40aa-b3dc-52988e3e5c86 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:18.198870 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1157/2335] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:18.213880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6c6f4-7b22-449b-826a-60b5140498a3 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:18.241711 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50681e21-5edd-4f27-bb9d-620309788472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:18.242406 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50681e21-5edd-4f27-bb9d-620309788472 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:18.242898 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50681e21-5edd-4f27-bb9d-620309788472 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:18.243185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50681e21-5edd-4f27-bb9d-620309788472 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:18.243959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50681e21-5edd-4f27-bb9d-620309788472 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:18.246010 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50681e21-5edd-4f27-bb9d-620309788472 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:18.276715 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1179/2336] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:50:18 2022] POST /identity/v3/projects => generated 385 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:50:18.280869 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50681e21-5edd-4f27-bb9d-620309788472 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:18.298665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6c6f4-7b22-449b-826a-60b5140498a3 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:18.356054 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50681e21-5edd-4f27-bb9d-620309788472 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:18.362015 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94882946-065f-436d-b00f-268389cbb17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:18.362587 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94882946-065f-436d-b00f-268389cbb17f admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:18.362974 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94882946-065f-436d-b00f-268389cbb17f admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:18.363327 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94882946-065f-436d-b00f-268389cbb17f admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:18.364128 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94882946-065f-436d-b00f-268389cbb17f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:18.366888 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94882946-065f-436d-b00f-268389cbb17f admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:18.371593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba33657-037e-4302-b2db-90d94f6d42bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:18.372243 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba33657-037e-4302-b2db-90d94f6d42bd service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:18.372519 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba33657-037e-4302-b2db-90d94f6d42bd service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:18.372861 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba33657-037e-4302-b2db-90d94f6d42bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:18.390921 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1180/2337] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:50:18 2022] POST /identity/v3/users => generated 299 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:50:18.394802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94882946-065f-436d-b00f-268389cbb17f admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:18.407989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94882946-065f-436d-b00f-268389cbb17f admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:18.413606 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b417e-e6b3-461f-9893-7b878edbaf93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:18.414560 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b417e-e6b3-461f-9893-7b878edbaf93 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:18.414840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b417e-e6b3-461f-9893-7b878edbaf93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:18.415117 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b417e-e6b3-461f-9893-7b878edbaf93 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:18.416064 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b417e-e6b3-461f-9893-7b878edbaf93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:18.418603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b417e-e6b3-461f-9893-7b878edbaf93 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:18.430520 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1181/2338] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:50:18.437664 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b417e-e6b3-461f-9893-7b878edbaf93 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:18.440500 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba33657-037e-4302-b2db-90d94f6d42bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:18.442176 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba33657-037e-4302-b2db-90d94f6d42bd service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:18.451273 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b417e-e6b3-461f-9893-7b878edbaf93 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:18.452200 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1158/2339] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:50:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:50:18.457364 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05d482-89f8-4b2b-b1a3-a17f201e297f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:18.458778 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05d482-89f8-4b2b-b1a3-a17f201e297f admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:18.459308 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05d482-89f8-4b2b-b1a3-a17f201e297f admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:18.459903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05d482-89f8-4b2b-b1a3-a17f201e297f admin admin] PATH_INFO: `/v3/projects/f9e2ae16aa4b42e89d26314a25d345a0/users/31801484334445a4a8bcf6fc6dfee4fc/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:18.481561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05d482-89f8-4b2b-b1a3-a17f201e297f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9e2ae16aa4b42e89d26314a25d345a0, user_id=31801484334445a4a8bcf6fc6dfee4fc, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:18.484898 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05d482-89f8-4b2b-b1a3-a17f201e297f admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:18.501724 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1182/2340] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:50:18 2022] PUT /identity/v3/projects/f9e2ae16aa4b42e89d26314a25d345a0/users/31801484334445a4a8bcf6fc6dfee4fc/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:18.505602 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba33657-037e-4302-b2db-90d94f6d42bd service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:18.524937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba33657-037e-4302-b2db-90d94f6d42bd service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:18.622378 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a70d6-f624-45a2-ba3b-863861791343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:18.623060 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a70d6-f624-45a2-ba3b-863861791343 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:18.623338 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a70d6-f624-45a2-ba3b-863861791343 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:18.623615 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a70d6-f624-45a2-ba3b-863861791343 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:18.624615 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042a70d6-f624-45a2-ba3b-863861791343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:18.626206 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042a70d6-f624-45a2-ba3b-863861791343 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:18.647360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1159/2341] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:50:18.651215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05d482-89f8-4b2b-b1a3-a17f201e297f admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:18.664510 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05d482-89f8-4b2b-b1a3-a17f201e297f admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:18.672417 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e3d29-3ff9-49e4-a566-da132835b47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:18.673601 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e3d29-3ff9-49e4-a566-da132835b47f admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:18.674027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e3d29-3ff9-49e4-a566-da132835b47f admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:18.674361 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e3d29-3ff9-49e4-a566-da132835b47f admin admin] PATH_INFO: `/v3/projects/f9e2ae16aa4b42e89d26314a25d345a0/users/31801484334445a4a8bcf6fc6dfee4fc/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:18.676687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e3d29-3ff9-49e4-a566-da132835b47f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9e2ae16aa4b42e89d26314a25d345a0, user_id=31801484334445a4a8bcf6fc6dfee4fc, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:18.679911 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e3d29-3ff9-49e4-a566-da132835b47f admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:18.691904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1183/2342] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:50:18 2022] PUT /identity/v3/projects/f9e2ae16aa4b42e89d26314a25d345a0/users/31801484334445a4a8bcf6fc6dfee4fc/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:20.252305 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0880850-5a8c-4dc2-960e-3a0ac6a63604 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:20.252872 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0880850-5a8c-4dc2-960e-3a0ac6a63604 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:20.253095 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0880850-5a8c-4dc2-960e-3a0ac6a63604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:20.275189 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-e0880850-5a8c-4dc2-960e-3a0ac6a63604 None None] MFA Rules not processed for user `17a673de26884004bc852ef76b18aa06`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:20.276808 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e0880850-5a8c-4dc2-960e-3a0ac6a63604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:20.348711 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1160/2343] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:21.339362 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e3d29-3ff9-49e4-a566-da132835b47f admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:21.507697 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e3d29-3ff9-49e4-a566-da132835b47f admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:21.529041 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60935391-6a83-46d2-aeea-2e5894930a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:21.530242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60935391-6a83-46d2-aeea-2e5894930a1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:21.530886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60935391-6a83-46d2-aeea-2e5894930a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:21.531358 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60935391-6a83-46d2-aeea-2e5894930a1f admin admin] PATH_INFO: `/v3/users/f57ea01b72cb4cbfb6c19959ffd2f318` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:21.536504 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60935391-6a83-46d2-aeea-2e5894930a1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f57ea01b72cb4cbfb6c19959ffd2f318)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:21.538776 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60935391-6a83-46d2-aeea-2e5894930a1f admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:21.657335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-60935391-6a83-46d2-aeea-2e5894930a1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f57ea01b72cb4cbfb6c19959ffd2f318'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:21.666331 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-60935391-6a83-46d2-aeea-2e5894930a1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f57ea01b72cb4cbfb6c19959ffd2f318'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:21.677444 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-60935391-6a83-46d2-aeea-2e5894930a1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f57ea01b72cb4cbfb6c19959ffd2f318'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:21.691251 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-60935391-6a83-46d2-aeea-2e5894930a1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f57ea01b72cb4cbfb6c19959ffd2f318'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:21.691970 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-60935391-6a83-46d2-aeea-2e5894930a1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f57ea01b72cb4cbfb6c19959ffd2f318'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:21.729610 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1184/2344] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:50:21 2022] DELETE /identity/v3/users/f57ea01b72cb4cbfb6c19959ffd2f318 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:21.734340 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897e13b-9c6e-43bd-a5ba-cac1644a15c2 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:21.734858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897e13b-9c6e-43bd-a5ba-cac1644a15c2 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:21.734979 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897e13b-9c6e-43bd-a5ba-cac1644a15c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:21.762042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-8897e13b-9c6e-43bd-a5ba-cac1644a15c2 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:21.764055 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-8897e13b-9c6e-43bd-a5ba-cac1644a15c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:21.855607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1161/2345] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:21.879191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60935391-6a83-46d2-aeea-2e5894930a1f admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:21.880750 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-60935391-6a83-46d2-aeea-2e5894930a1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:22.053759 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60935391-6a83-46d2-aeea-2e5894930a1f admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:22.057677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b44d8-7450-45aa-9055-627c0b517277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:22.058459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b44d8-7450-45aa-9055-627c0b517277 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:22.058828 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b44d8-7450-45aa-9055-627c0b517277 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:22.059190 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b44d8-7450-45aa-9055-627c0b517277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:22.090057 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b44d8-7450-45aa-9055-627c0b517277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:22.091893 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b44d8-7450-45aa-9055-627c0b517277 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:22.104178 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1185/2346] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:50:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:50:22.477034 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897e13b-9c6e-43bd-a5ba-cac1644a15c2 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:22.478935 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-8897e13b-9c6e-43bd-a5ba-cac1644a15c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:22.502284 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897e13b-9c6e-43bd-a5ba-cac1644a15c2 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:22.509042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3251643-f53b-4559-95c0-44cf4260a96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:22.510105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3251643-f53b-4559-95c0-44cf4260a96a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:22.514010 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3251643-f53b-4559-95c0-44cf4260a96a admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:22.514289 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3251643-f53b-4559-95c0-44cf4260a96a admin admin] PATH_INFO: `/v3/projects/5a7d53bd35904af49f40686d29a270be` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:22.517473 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3251643-f53b-4559-95c0-44cf4260a96a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a7d53bd35904af49f40686d29a270be)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:22.520515 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3251643-f53b-4559-95c0-44cf4260a96a admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:22.574825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c3251643-f53b-4559-95c0-44cf4260a96a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a7d53bd35904af49f40686d29a270be', 'request_id': 'req-c3251643-f53b-4559-95c0-44cf4260a96a'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:22.582806 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c3251643-f53b-4559-95c0-44cf4260a96a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a7d53bd35904af49f40686d29a270be', 'request_id': 'req-c3251643-f53b-4559-95c0-44cf4260a96a'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:22.589071 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c3251643-f53b-4559-95c0-44cf4260a96a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a7d53bd35904af49f40686d29a270be', 'request_id': 'req-c3251643-f53b-4559-95c0-44cf4260a96a'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:22.608641 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c3251643-f53b-4559-95c0-44cf4260a96a admin admin] The token cache is being invalidate because project 5a7d53bd35904af49f40686d29a270be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:50:22.609113 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c3251643-f53b-4559-95c0-44cf4260a96a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:22.609896 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c3251643-f53b-4559-95c0-44cf4260a96a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:22.611633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1162/2347] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:50:22 2022] DELETE /identity/v3/projects/5a7d53bd35904af49f40686d29a270be => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:22.626966 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67935f17-cf62-4e2e-b233-5bea8d2794eb None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:22.626966 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67935f17-cf62-4e2e-b233-5bea8d2794eb None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:22.626966 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67935f17-cf62-4e2e-b233-5bea8d2794eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:22.641943 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-67935f17-cf62-4e2e-b233-5bea8d2794eb None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:22.650944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-67935f17-cf62-4e2e-b233-5bea8d2794eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:22.741263 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1186/2348] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:22.749521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b86ad2-5bea-49e3-9cda-54de57c871d5 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:22.750122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b86ad2-5bea-49e3-9cda-54de57c871d5 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:22.750122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b86ad2-5bea-49e3-9cda-54de57c871d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:22.773910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-18b86ad2-5bea-49e3-9cda-54de57c871d5 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:22.776428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-18b86ad2-5bea-49e3-9cda-54de57c871d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:22.792362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1163/2349] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:22.797365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67935f17-cf62-4e2e-b233-5bea8d2794eb None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:22.808604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67935f17-cf62-4e2e-b233-5bea8d2794eb None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:22.814874 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299cfa6-44ef-4200-b0bf-b5831bcc3f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:22.815554 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299cfa6-44ef-4200-b0bf-b5831bcc3f7d admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:22.815829 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299cfa6-44ef-4200-b0bf-b5831bcc3f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:22.816098 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299cfa6-44ef-4200-b0bf-b5831bcc3f7d admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:22.816914 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7299cfa6-44ef-4200-b0bf-b5831bcc3f7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 09:50:22.817215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7299cfa6-44ef-4200-b0bf-b5831bcc3f7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:22.818945 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7299cfa6-44ef-4200-b0bf-b5831bcc3f7d admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:22.838952 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1187/2350] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 09:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 09:50:22.839696 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b86ad2-5bea-49e3-9cda-54de57c871d5 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:22.853835 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b86ad2-5bea-49e3-9cda-54de57c871d5 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:22.860930 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05bcf2c-66fa-468c-8555-4c833602931a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:22.861833 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bcf2c-66fa-468c-8555-4c833602931a admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:22.862840 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bcf2c-66fa-468c-8555-4c833602931a admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:22.863187 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bcf2c-66fa-468c-8555-4c833602931a admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:22.864025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05bcf2c-66fa-468c-8555-4c833602931a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:22.866243 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05bcf2c-66fa-468c-8555-4c833602931a admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:22.963445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1164/2351] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 09:50:22 2022] POST /identity/v3/projects => generated 397 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 09:50:22.970850 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299cfa6-44ef-4200-b0bf-b5831bcc3f7d admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:23.007328 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299cfa6-44ef-4200-b0bf-b5831bcc3f7d admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:23.122560 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0eb299-eb9b-40b4-aa99-f32c6b48bf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:23.123449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0eb299-eb9b-40b4-aa99-f32c6b48bf9a admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:23.123557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0eb299-eb9b-40b4-aa99-f32c6b48bf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:23.123783 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0eb299-eb9b-40b4-aa99-f32c6b48bf9a admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:23.124485 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0eb299-eb9b-40b4-aa99-f32c6b48bf9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:23.126932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0eb299-eb9b-40b4-aa99-f32c6b48bf9a admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:23.162804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05bcf2c-66fa-468c-8555-4c833602931a admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:23.163578 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1188/2352] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 09:50:22 2022] POST /identity/v3/users => generated 299 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 09:50:23.182701 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05bcf2c-66fa-468c-8555-4c833602931a admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:23.205949 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed219102-4bbb-4a79-a468-ce9149c35368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:23.207413 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed219102-4bbb-4a79-a468-ce9149c35368 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:23.208188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed219102-4bbb-4a79-a468-ce9149c35368 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:23.209886 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed219102-4bbb-4a79-a468-ce9149c35368 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:23.210817 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed219102-4bbb-4a79-a468-ce9149c35368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:23.213412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed219102-4bbb-4a79-a468-ce9149c35368 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:23.230067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0eb299-eb9b-40b4-aa99-f32c6b48bf9a admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:23.231038 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1165/2353] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 09:50:23.245342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0eb299-eb9b-40b4-aa99-f32c6b48bf9a admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:23.251553 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597048b-bcf9-4c7a-9c81-9e16c4ba65af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:23.253252 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597048b-bcf9-4c7a-9c81-9e16c4ba65af admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:23.253732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597048b-bcf9-4c7a-9c81-9e16c4ba65af admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:23.254168 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597048b-bcf9-4c7a-9c81-9e16c4ba65af admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:23.255585 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a597048b-bcf9-4c7a-9c81-9e16c4ba65af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:23.258162 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a597048b-bcf9-4c7a-9c81-9e16c4ba65af admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:23.277330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed219102-4bbb-4a79-a468-ce9149c35368 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:23.281037 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1189/2354] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 09:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 09:50:23.289753 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed219102-4bbb-4a79-a468-ce9149c35368 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:23.302924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277172b-8c1a-4b50-8cc5-24bcd7ca9ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:23.306033 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277172b-8c1a-4b50-8cc5-24bcd7ca9ebb admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:23.306984 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277172b-8c1a-4b50-8cc5-24bcd7ca9ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:23.307430 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277172b-8c1a-4b50-8cc5-24bcd7ca9ebb admin admin] PATH_INFO: `/v3/projects/8ef6ca17768a4fe89e793d4699e7f4f4/users/10acf2f3347e498e9cf6e92a41b13f1a/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:23.324717 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277172b-8c1a-4b50-8cc5-24bcd7ca9ebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ef6ca17768a4fe89e793d4699e7f4f4, user_id=10acf2f3347e498e9cf6e92a41b13f1a, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:23.327522 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277172b-8c1a-4b50-8cc5-24bcd7ca9ebb admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:23.372449 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1166/2355] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 09:50:23 2022] PUT /identity/v3/projects/8ef6ca17768a4fe89e793d4699e7f4f4/users/10acf2f3347e498e9cf6e92a41b13f1a/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:23.373586 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545e89b-b4d3-4a70-8c74-7852fa91fe8e None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:23.373877 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545e89b-b4d3-4a70-8c74-7852fa91fe8e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:23.374153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545e89b-b4d3-4a70-8c74-7852fa91fe8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:23.436770 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-c545e89b-b4d3-4a70-8c74-7852fa91fe8e None None] MFA Rules not processed for user `10acf2f3347e498e9cf6e92a41b13f1a`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:23.440787 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c545e89b-b4d3-4a70-8c74-7852fa91fe8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:23.733171 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1190/2356] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:50:23.741686 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277172b-8c1a-4b50-8cc5-24bcd7ca9ebb admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:23.743729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a277172b-8c1a-4b50-8cc5-24bcd7ca9ebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:23.839078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277172b-8c1a-4b50-8cc5-24bcd7ca9ebb admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:23.850074 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74aa460-bbd7-4e44-9f11-56c3d135b8e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:23.851583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74aa460-bbd7-4e44-9f11-56c3d135b8e6 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:23.851864 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74aa460-bbd7-4e44-9f11-56c3d135b8e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:23.852451 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74aa460-bbd7-4e44-9f11-56c3d135b8e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:23.877616 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74aa460-bbd7-4e44-9f11-56c3d135b8e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:23.880319 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74aa460-bbd7-4e44-9f11-56c3d135b8e6 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:23.893831 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1167/2357] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:50:23 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:50:23.980120 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545e89b-b4d3-4a70-8c74-7852fa91fe8e None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:23.981834 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c545e89b-b4d3-4a70-8c74-7852fa91fe8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:24.151085 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545e89b-b4d3-4a70-8c74-7852fa91fe8e None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:24.156573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0218d6f-b07f-4ad6-9a50-ec1b8bebd1eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9e458c2c5f4ed1b3eddd84c2ec1703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:24.158450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0218d6f-b07f-4ad6-9a50-ec1b8bebd1eb service placement] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:24.158876 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0218d6f-b07f-4ad6-9a50-ec1b8bebd1eb service placement] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:24.159152 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0218d6f-b07f-4ad6-9a50-ec1b8bebd1eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:24.162148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b0218d6f-b07f-4ad6-9a50-ec1b8bebd1eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:24.192858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0218d6f-b07f-4ad6-9a50-ec1b8bebd1eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:24.194774 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0218d6f-b07f-4ad6-9a50-ec1b8bebd1eb service placement] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:24.206562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1191/2358] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:50:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:50:25.763019 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb77676-7b27-4a5c-9bca-f56489107fdf None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:25.764137 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb77676-7b27-4a5c-9bca-f56489107fdf None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:25.764606 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb77676-7b27-4a5c-9bca-f56489107fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:25.803024 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-fcb77676-7b27-4a5c-9bca-f56489107fdf None None] MFA Rules not processed for user `31801484334445a4a8bcf6fc6dfee4fc`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:25.804562 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-fcb77676-7b27-4a5c-9bca-f56489107fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:25.922419 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1168/2359] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:25.932380 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0218d6f-b07f-4ad6-9a50-ec1b8bebd1eb service placement] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:25.935029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b0218d6f-b07f-4ad6-9a50-ec1b8bebd1eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:26.024044 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0218d6f-b07f-4ad6-9a50-ec1b8bebd1eb service placement] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:26.027826 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7413c9d-1322-4a97-8d5e-838c6c99ee54 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fd398c547554629927d2a80b3a8dee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:26.028576 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7413c9d-1322-4a97-8d5e-838c6c99ee54 service glance] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:26.028973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7413c9d-1322-4a97-8d5e-838c6c99ee54 service glance] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:26.029363 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7413c9d-1322-4a97-8d5e-838c6c99ee54 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:26.031224 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-f7413c9d-1322-4a97-8d5e-838c6c99ee54 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:26.101796 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7413c9d-1322-4a97-8d5e-838c6c99ee54 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:26.103371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7413c9d-1322-4a97-8d5e-838c6c99ee54 service glance] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:26.116710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1192/2360] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:50:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:50:26.235431 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18869f-ff79-4a9a-8e10-56957b636a73 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:26.235719 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18869f-ff79-4a9a-8e10-56957b636a73 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:26.235992 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18869f-ff79-4a9a-8e10-56957b636a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:26.298743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-3a18869f-ff79-4a9a-8e10-56957b636a73 None None] MFA Rules not processed for user `af62b2d3f73f482b8db4173af33752c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:26.299981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-3a18869f-ff79-4a9a-8e10-56957b636a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:26.372045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1169/2361] 10.222.0.26 () {58 vars in 1009 bytes} [Mon May 9 09:50:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:26.413912 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7413c9d-1322-4a97-8d5e-838c6c99ee54 service glance] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:26.415560 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-f7413c9d-1322-4a97-8d5e-838c6c99ee54 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:26.511564 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7413c9d-1322-4a97-8d5e-838c6c99ee54 service glance] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:26.516656 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8867ce-7f9a-4fcb-abef-99a8d9561b5b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380e04358f2349aa912d6a97518877cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:26.517323 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8867ce-7f9a-4fcb-abef-99a8d9561b5b service swift] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:26.517602 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8867ce-7f9a-4fcb-abef-99a8d9561b5b service swift] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:26.517875 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8867ce-7f9a-4fcb-abef-99a8d9561b5b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:26.525861 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8867ce-7f9a-4fcb-abef-99a8d9561b5b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:26.527449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8867ce-7f9a-4fcb-abef-99a8d9561b5b service swift] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:26.535450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1193/2362] 10.222.0.26 () {60 vars in 1384 bytes} [Mon May 9 09:50:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 09:50:33.585827 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18869f-ff79-4a9a-8e10-56957b636a73 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:33.689382 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18869f-ff79-4a9a-8e10-56957b636a73 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:33.694238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57cb83-f7ed-4a82-8ae3-517be6861218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:33.695357 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57cb83-f7ed-4a82-8ae3-517be6861218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:33.695986 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57cb83-f7ed-4a82-8ae3-517be6861218 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:33.696464 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57cb83-f7ed-4a82-8ae3-517be6861218 admin admin] PATH_INFO: `/v3/users/10acf2f3347e498e9cf6e92a41b13f1a` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:33.697964 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57cb83-f7ed-4a82-8ae3-517be6861218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10acf2f3347e498e9cf6e92a41b13f1a)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:33.700108 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57cb83-f7ed-4a82-8ae3-517be6861218 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:33.757187 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-fc57cb83-f7ed-4a82-8ae3-517be6861218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10acf2f3347e498e9cf6e92a41b13f1a'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:33.765281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-fc57cb83-f7ed-4a82-8ae3-517be6861218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10acf2f3347e498e9cf6e92a41b13f1a'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:33.779558 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-fc57cb83-f7ed-4a82-8ae3-517be6861218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10acf2f3347e498e9cf6e92a41b13f1a'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:33.780916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-fc57cb83-f7ed-4a82-8ae3-517be6861218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10acf2f3347e498e9cf6e92a41b13f1a'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:33.781965 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-fc57cb83-f7ed-4a82-8ae3-517be6861218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10acf2f3347e498e9cf6e92a41b13f1a'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:33.835548 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1170/2363] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 09:50:33 2022] DELETE /identity/v3/users/10acf2f3347e498e9cf6e92a41b13f1a => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:33.837293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b288c79-c264-44fa-848a-fc979e6684eb None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:33.837915 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b288c79-c264-44fa-848a-fc979e6684eb None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:33.837915 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b288c79-c264-44fa-848a-fc979e6684eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:33.858454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-5b288c79-c264-44fa-848a-fc979e6684eb None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:33.859757 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-5b288c79-c264-44fa-848a-fc979e6684eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:33.928380 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1194/2364] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:33.938506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57cb83-f7ed-4a82-8ae3-517be6861218 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:33.939930 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-fc57cb83-f7ed-4a82-8ae3-517be6861218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:34.021217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57cb83-f7ed-4a82-8ae3-517be6861218 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:34.026202 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2ca93-aa50-415e-9d63-cc422f8b7d52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:34.026948 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2ca93-aa50-415e-9d63-cc422f8b7d52 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:34.027288 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2ca93-aa50-415e-9d63-cc422f8b7d52 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:34.027623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2ca93-aa50-415e-9d63-cc422f8b7d52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:34.039071 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae2ca93-aa50-415e-9d63-cc422f8b7d52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:34.040695 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae2ca93-aa50-415e-9d63-cc422f8b7d52 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:34.051937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1171/2365] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:50:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:50:34.472353 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b288c79-c264-44fa-848a-fc979e6684eb None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:34.474746 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-5b288c79-c264-44fa-848a-fc979e6684eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:34.509598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b288c79-c264-44fa-848a-fc979e6684eb None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:34.517189 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9109ac-046a-46da-87a2-959b13a73de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:34.518345 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9109ac-046a-46da-87a2-959b13a73de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:34.518825 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9109ac-046a-46da-87a2-959b13a73de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:34.519235 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9109ac-046a-46da-87a2-959b13a73de5 admin admin] PATH_INFO: `/v3/projects/8ef6ca17768a4fe89e793d4699e7f4f4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:34.521981 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9109ac-046a-46da-87a2-959b13a73de5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ef6ca17768a4fe89e793d4699e7f4f4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:34.526630 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9109ac-046a-46da-87a2-959b13a73de5 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:34.587990 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bb9109ac-046a-46da-87a2-959b13a73de5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ef6ca17768a4fe89e793d4699e7f4f4', 'request_id': 'req-bb9109ac-046a-46da-87a2-959b13a73de5'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:34.595784 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bb9109ac-046a-46da-87a2-959b13a73de5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ef6ca17768a4fe89e793d4699e7f4f4', 'request_id': 'req-bb9109ac-046a-46da-87a2-959b13a73de5'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:34.603590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bb9109ac-046a-46da-87a2-959b13a73de5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ef6ca17768a4fe89e793d4699e7f4f4', 'request_id': 'req-bb9109ac-046a-46da-87a2-959b13a73de5'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:34.618485 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bb9109ac-046a-46da-87a2-959b13a73de5 admin admin] The token cache is being invalidate because project 8ef6ca17768a4fe89e793d4699e7f4f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 09:50:34.619047 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bb9109ac-046a-46da-87a2-959b13a73de5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:34.619729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bb9109ac-046a-46da-87a2-959b13a73de5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 09:50:34.624757 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1195/2366] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 09:50:34 2022] DELETE /identity/v3/projects/8ef6ca17768a4fe89e793d4699e7f4f4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 09:50:36.276086 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df61b1d-1778-4fdb-8b7f-3238ed945b56 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:36.276759 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df61b1d-1778-4fdb-8b7f-3238ed945b56 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:36.276759 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df61b1d-1778-4fdb-8b7f-3238ed945b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:36.295102 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-2df61b1d-1778-4fdb-8b7f-3238ed945b56 None None] MFA Rules not processed for user `cba3e3a904ca41b78e518699f88e315e`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:36.296466 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-2df61b1d-1778-4fdb-8b7f-3238ed945b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:36.352970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1172/2367] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:50:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:41.002569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f142c-874f-41b2-98a2-6db9194784e5 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:41.003135 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f142c-874f-41b2-98a2-6db9194784e5 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:41.003135 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f142c-874f-41b2-98a2-6db9194784e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:41.021179 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-883f142c-874f-41b2-98a2-6db9194784e5 None None] MFA Rules not processed for user `af62b2d3f73f482b8db4173af33752c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:50:41.022533 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-883f142c-874f-41b2-98a2-6db9194784e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:41.085934 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1196/2368] 10.222.0.26 () {58 vars in 1009 bytes} [Mon May 9 09:50:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:50:41.099614 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df61b1d-1778-4fdb-8b7f-3238ed945b56 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:50:41.101022 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-2df61b1d-1778-4fdb-8b7f-3238ed945b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:50:41.180140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df61b1d-1778-4fdb-8b7f-3238ed945b56 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:50:41.186300 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d685ea-e65e-43d3-a0b7-ae3de86dc918 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380e04358f2349aa912d6a97518877cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:50:41.187104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d685ea-e65e-43d3-a0b7-ae3de86dc918 service swift] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:50:41.187472 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d685ea-e65e-43d3-a0b7-ae3de86dc918 service swift] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:50:41.187833 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d685ea-e65e-43d3-a0b7-ae3de86dc918 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:50:41.197132 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d685ea-e65e-43d3-a0b7-ae3de86dc918 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:50:41.198894 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d685ea-e65e-43d3-a0b7-ae3de86dc918 service swift] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:50:41.207212 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1173/2369] 10.222.0.26 () {60 vars in 1384 bytes} [Mon May 9 09:50:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 09:51:20.640010 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f142c-874f-41b2-98a2-6db9194784e5 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:51:20.642157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-883f142c-874f-41b2-98a2-6db9194784e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:51:20.713317 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f142c-874f-41b2-98a2-6db9194784e5 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:51:20.717933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6335fa8-05d4-43d5-9d61-5fedfb319fa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:51:20.718599 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6335fa8-05d4-43d5-9d61-5fedfb319fa4 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:51:20.718874 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6335fa8-05d4-43d5-9d61-5fedfb319fa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:51:20.719134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6335fa8-05d4-43d5-9d61-5fedfb319fa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:51:20.721406 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b6335fa8-05d4-43d5-9d61-5fedfb319fa4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:51:20.799914 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6335fa8-05d4-43d5-9d61-5fedfb319fa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:51:20.801724 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6335fa8-05d4-43d5-9d61-5fedfb319fa4 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:51:20.813081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1197/2370] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:51:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:51:36.330734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b812b-2dd3-438a-9ef4-7f69b5ba51e4 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:51:36.331604 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b812b-2dd3-438a-9ef4-7f69b5ba51e4 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:51:36.331604 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b812b-2dd3-438a-9ef4-7f69b5ba51e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:51:36.354015 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-5c4b812b-2dd3-438a-9ef4-7f69b5ba51e4 None None] MFA Rules not processed for user `af62b2d3f73f482b8db4173af33752c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:51:36.355526 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-5c4b812b-2dd3-438a-9ef4-7f69b5ba51e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:51:36.363431 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1174/2371] 10.222.0.26 () {58 vars in 1009 bytes} [Mon May 9 09:51:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:51:36.386716 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6335fa8-05d4-43d5-9d61-5fedfb319fa4 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:51:36.399064 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6335fa8-05d4-43d5-9d61-5fedfb319fa4 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:51:36.404533 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc159e-2079-44d1-8914-5ea357f6ddf8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380e04358f2349aa912d6a97518877cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:51:36.405240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc159e-2079-44d1-8914-5ea357f6ddf8 service swift] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:51:36.405517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc159e-2079-44d1-8914-5ea357f6ddf8 service swift] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:51:36.405792 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc159e-2079-44d1-8914-5ea357f6ddf8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:51:36.414314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc159e-2079-44d1-8914-5ea357f6ddf8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:51:36.415959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc159e-2079-44d1-8914-5ea357f6ddf8 service swift] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:51:36.424975 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1198/2372] 10.222.0.26 () {60 vars in 1384 bytes} [Mon May 9 09:51:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 09:51:56.258975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de765a-fad9-4977-be54-f8c02a9677ad None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:51:56.259697 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de765a-fad9-4977-be54-f8c02a9677ad None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:51:56.259697 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de765a-fad9-4977-be54-f8c02a9677ad None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:51:56.260999 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1175/2373] 10.222.0.26 () {54 vars in 921 bytes} [Mon May 9 09:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:51:56.267618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a35cfa-19e4-48d1-b43f-a62e15f81669 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:51:56.268230 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a35cfa-19e4-48d1-b43f-a62e15f81669 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:51:56.268230 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a35cfa-19e4-48d1-b43f-a62e15f81669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:51:56.333034 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-e2a35cfa-19e4-48d1-b43f-a62e15f81669 None None] MFA Rules not processed for user `6362303cb918486c8525225914332ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:51:56.334319 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e2a35cfa-19e4-48d1-b43f-a62e15f81669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:51:56.431794 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1199/2374] 10.222.0.26 () {58 vars in 1018 bytes} [Mon May 9 09:51:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:51:56.450496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de765a-fad9-4977-be54-f8c02a9677ad None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:51:56.452218 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-05de765a-fad9-4977-be54-f8c02a9677ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:51:56.520624 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de765a-fad9-4977-be54-f8c02a9677ad None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:51:56.524789 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e65ee-183d-49e8-9aae-20f5ee75da42 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fd398c547554629927d2a80b3a8dee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:51:56.525544 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e65ee-183d-49e8-9aae-20f5ee75da42 service glance] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:51:56.525899 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e65ee-183d-49e8-9aae-20f5ee75da42 service glance] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:51:56.526256 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e65ee-183d-49e8-9aae-20f5ee75da42 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:51:56.538679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e65ee-183d-49e8-9aae-20f5ee75da42 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:51:56.540232 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e65ee-183d-49e8-9aae-20f5ee75da42 service glance] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:51:56.553898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1176/2375] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:51:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:51:56.644827 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052957a-29f1-463f-a44d-2a0f58e6dce9 None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:51:56.645129 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052957a-29f1-463f-a44d-2a0f58e6dce9 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:51:56.645389 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052957a-29f1-463f-a44d-2a0f58e6dce9 None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:51:56.647814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1200/2376] 10.222.0.26 () {54 vars in 921 bytes} [Mon May 9 09:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:51:56.652226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f36d0-f8bf-4653-b6e3-a66b31121f11 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:51:56.652506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f36d0-f8bf-4653-b6e3-a66b31121f11 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:51:56.652804 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f36d0-f8bf-4653-b6e3-a66b31121f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:51:56.671356 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-af9f36d0-f8bf-4653-b6e3-a66b31121f11 None None] MFA Rules not processed for user `0f0d3f4264a04093894d6bc5c5c45c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:51:56.672580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-af9f36d0-f8bf-4653-b6e3-a66b31121f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:51:56.679235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1177/2377] 10.222.0.26 () {58 vars in 1018 bytes} [Mon May 9 09:51:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 09:51:56.692813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052957a-29f1-463f-a44d-2a0f58e6dce9 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:51:56.705713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052957a-29f1-463f-a44d-2a0f58e6dce9 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:51:56.709802 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c53cea-deec-4011-ba1a-b6da1795fb55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:51:56.710461 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c53cea-deec-4011-ba1a-b6da1795fb55 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:51:56.710744 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c53cea-deec-4011-ba1a-b6da1795fb55 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:51:56.711016 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c53cea-deec-4011-ba1a-b6da1795fb55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:51:56.719288 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c53cea-deec-4011-ba1a-b6da1795fb55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:51:56.720770 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c53cea-deec-4011-ba1a-b6da1795fb55 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:51:56.728531 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1201/2378] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:51:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:51:56.828730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750e3b8-10d9-43fd-8133-3ab546eea593 None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:51:56.829027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750e3b8-10d9-43fd-8133-3ab546eea593 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:51:56.829290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750e3b8-10d9-43fd-8133-3ab546eea593 None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:51:56.830690 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1178/2379] 10.222.0.26 () {54 vars in 921 bytes} [Mon May 9 09:51:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 09:51:56.836576 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b78c0-473b-4544-b751-02d4899bef94 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:51:56.836889 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b78c0-473b-4544-b751-02d4899bef94 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:51:56.837316 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b78c0-473b-4544-b751-02d4899bef94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:51:56.862789 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-e15b78c0-473b-4544-b751-02d4899bef94 None None] MFA Rules not processed for user `6362303cb918486c8525225914332ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:51:56.864235 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e15b78c0-473b-4544-b751-02d4899bef94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:51:56.876645 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1202/2380] 10.222.0.26 () {58 vars in 1018 bytes} [Mon May 9 09:51:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:51:56.888049 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750e3b8-10d9-43fd-8133-3ab546eea593 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:51:56.900188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750e3b8-10d9-43fd-8133-3ab546eea593 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:51:56.904176 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f399c0-761b-4144-978c-8fb0d839cef0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:51:56.904889 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f399c0-761b-4144-978c-8fb0d839cef0 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:51:56.905169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f399c0-761b-4144-978c-8fb0d839cef0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:51:56.905473 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f399c0-761b-4144-978c-8fb0d839cef0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:51:56.916489 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f399c0-761b-4144-978c-8fb0d839cef0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:51:56.918005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f399c0-761b-4144-978c-8fb0d839cef0 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:51:56.931489 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1179/2381] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:51:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:52:02.726536 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a886c-19e3-4d0d-9877-46878655af0d None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:52:02.727338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a886c-19e3-4d0d-9877-46878655af0d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:52:02.727338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a886c-19e3-4d0d-9877-46878655af0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:52:02.747398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-c55a886c-19e3-4d0d-9877-46878655af0d None None] MFA Rules not processed for user `af62b2d3f73f482b8db4173af33752c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:52:02.748616 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c55a886c-19e3-4d0d-9877-46878655af0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:52:02.757459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1203/2382] 10.222.0.26 () {58 vars in 1009 bytes} [Mon May 9 09:52:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:52:02.770060 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f399c0-761b-4144-978c-8fb0d839cef0 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:52:02.782450 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f399c0-761b-4144-978c-8fb0d839cef0 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:52:02.787463 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a13f4-f2f0-42ab-ba62-bd58dfae5571 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380e04358f2349aa912d6a97518877cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:52:02.788115 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a13f4-f2f0-42ab-ba62-bd58dfae5571 service swift] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:52:02.788386 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a13f4-f2f0-42ab-ba62-bd58dfae5571 service swift] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:52:02.788653 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a13f4-f2f0-42ab-ba62-bd58dfae5571 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:52:02.797623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a13f4-f2f0-42ab-ba62-bd58dfae5571 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:52:02.799270 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a13f4-f2f0-42ab-ba62-bd58dfae5571 service swift] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:52:02.808321 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1180/2383] 10.222.0.26 () {60 vars in 1384 bytes} [Mon May 9 09:52:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 09:52:17.697951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a886c-19e3-4d0d-9877-46878655af0d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:52:17.706208 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a13f4-f2f0-42ab-ba62-bd58dfae5571 service swift] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:52:17.708403 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-df3a13f4-f2f0-42ab-ba62-bd58dfae5571 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:52:17.708572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a886c-19e3-4d0d-9877-46878655af0d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:52:17.712030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83e00d-ebf4-4c4a-a671-e6413c2e8c93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:52:17.712707 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83e00d-ebf4-4c4a-a671-e6413c2e8c93 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:52:17.712977 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83e00d-ebf4-4c4a-a671-e6413c2e8c93 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:52:17.713379 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83e00d-ebf4-4c4a-a671-e6413c2e8c93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:52:17.715200 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-5d83e00d-ebf4-4c4a-a671-e6413c2e8c93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:52:17.721319 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a13f4-f2f0-42ab-ba62-bd58dfae5571 service swift] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:52:17.725221 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82085e53-a472-4541-94ac-20088d7a4621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:52:17.725779 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82085e53-a472-4541-94ac-20088d7a4621 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:52:17.726422 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82085e53-a472-4541-94ac-20088d7a4621 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:52:17.726422 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82085e53-a472-4541-94ac-20088d7a4621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:52:17.728193 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-82085e53-a472-4541-94ac-20088d7a4621 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:52:17.732916 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83e00d-ebf4-4c4a-a671-e6413c2e8c93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:52:17.734765 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83e00d-ebf4-4c4a-a671-e6413c2e8c93 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:52:17.743436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82085e53-a472-4541-94ac-20088d7a4621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:52:17.744919 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82085e53-a472-4541-94ac-20088d7a4621 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:52:17.746409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1204/2384] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:52:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:52:17.759943 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1181/2385] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:52:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:52:24.807542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83e00d-ebf4-4c4a-a671-e6413c2e8c93 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:52:24.809078 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-5d83e00d-ebf4-4c4a-a671-e6413c2e8c93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:52:24.823200 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83e00d-ebf4-4c4a-a671-e6413c2e8c93 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:52:24.830297 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804e41f-74ea-44d8-a8fe-39b69f59d224 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:52:24.830942 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804e41f-74ea-44d8-a8fe-39b69f59d224 service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:52:24.831210 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804e41f-74ea-44d8-a8fe-39b69f59d224 service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:52:24.831473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804e41f-74ea-44d8-a8fe-39b69f59d224 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:52:24.833478 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-2804e41f-74ea-44d8-a8fe-39b69f59d224 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:52:24.893028 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804e41f-74ea-44d8-a8fe-39b69f59d224 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:52:24.894670 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804e41f-74ea-44d8-a8fe-39b69f59d224 service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:52:24.906510 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1205/2386] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:52:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:52:32.939885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e98d39-694d-4814-b4e6-9b131b1c8cb5 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:52:32.940581 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e98d39-694d-4814-b4e6-9b131b1c8cb5 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:52:32.940581 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e98d39-694d-4814-b4e6-9b131b1c8cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:52:32.960811 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-92e98d39-694d-4814-b4e6-9b131b1c8cb5 None None] MFA Rules not processed for user `17a673de26884004bc852ef76b18aa06`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:52:32.962027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-92e98d39-694d-4814-b4e6-9b131b1c8cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:52:33.019697 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1182/2387] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:52:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:52:33.269814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804e41f-74ea-44d8-a8fe-39b69f59d224 service nova] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:52:33.280791 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804e41f-74ea-44d8-a8fe-39b69f59d224 service nova] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:52:33.285091 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8495250-d7ef-42c2-b0a9-1983a4e9f198 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:52:33.285736 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8495250-d7ef-42c2-b0a9-1983a4e9f198 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:52:33.286012 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8495250-d7ef-42c2-b0a9-1983a4e9f198 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:52:33.286280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8495250-d7ef-42c2-b0a9-1983a4e9f198 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:52:33.288164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b8495250-d7ef-42c2-b0a9-1983a4e9f198 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:52:33.298213 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8495250-d7ef-42c2-b0a9-1983a4e9f198 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:52:33.299714 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8495250-d7ef-42c2-b0a9-1983a4e9f198 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:52:33.307997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1206/2388] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:52:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:52:33.632938 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e98d39-694d-4814-b4e6-9b131b1c8cb5 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:52:33.634509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-92e98d39-694d-4814-b4e6-9b131b1c8cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:52:33.705907 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e98d39-694d-4814-b4e6-9b131b1c8cb5 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:52:33.711352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35c425-396f-473a-95ec-e57265f99025 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9e458c2c5f4ed1b3eddd84c2ec1703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:52:33.711999 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35c425-396f-473a-95ec-e57265f99025 service placement] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:52:33.712272 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35c425-396f-473a-95ec-e57265f99025 service placement] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:52:33.712520 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35c425-396f-473a-95ec-e57265f99025 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:52:33.714422 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-2f35c425-396f-473a-95ec-e57265f99025 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:52:33.725921 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35c425-396f-473a-95ec-e57265f99025 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:52:33.727496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35c425-396f-473a-95ec-e57265f99025 service placement] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:52:33.737652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1183/2389] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:52:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:52:37.387283 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8495250-d7ef-42c2-b0a9-1983a4e9f198 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:52:37.397270 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8495250-d7ef-42c2-b0a9-1983a4e9f198 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:52:37.402134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11e905-2b89-426e-9787-0910a877439f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:52:37.402805 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11e905-2b89-426e-9787-0910a877439f service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:52:37.403074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11e905-2b89-426e-9787-0910a877439f service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:52:37.403339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11e905-2b89-426e-9787-0910a877439f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:52:37.405438 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-6b11e905-2b89-426e-9787-0910a877439f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:52:37.416589 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11e905-2b89-426e-9787-0910a877439f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:52:37.418088 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11e905-2b89-426e-9787-0910a877439f service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:52:37.426627 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1207/2390] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:52:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:52:38.290552 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43672e29-42f4-4d88-a8c2-844dbbfd7c1a None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:52:38.291094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43672e29-42f4-4d88-a8c2-844dbbfd7c1a None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:52:38.291094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43672e29-42f4-4d88-a8c2-844dbbfd7c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:52:38.310560 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-43672e29-42f4-4d88-a8c2-844dbbfd7c1a None None] MFA Rules not processed for user `af62b2d3f73f482b8db4173af33752c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:52:38.311946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-43672e29-42f4-4d88-a8c2-844dbbfd7c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:52:38.320586 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1184/2391] 10.222.0.26 () {58 vars in 967 bytes} [Mon May 9 09:52:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:52:38.344709 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11e905-2b89-426e-9787-0910a877439f service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:52:38.357689 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11e905-2b89-426e-9787-0910a877439f service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:52:38.363051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788859b8-2ea8-4fb0-be2a-776455e8b33d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380e04358f2349aa912d6a97518877cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:52:38.363694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788859b8-2ea8-4fb0-be2a-776455e8b33d service swift] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:52:38.363967 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788859b8-2ea8-4fb0-be2a-776455e8b33d service swift] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:52:38.364252 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788859b8-2ea8-4fb0-be2a-776455e8b33d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:52:38.372528 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788859b8-2ea8-4fb0-be2a-776455e8b33d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:52:38.374123 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788859b8-2ea8-4fb0-be2a-776455e8b33d service swift] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:52:38.381945 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1208/2392] 10.222.0.26 () {60 vars in 1384 bytes} [Mon May 9 09:52:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 09:52:59.987229 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43672e29-42f4-4d88-a8c2-844dbbfd7c1a None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:52:59.988875 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-43672e29-42f4-4d88-a8c2-844dbbfd7c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:53:00.005916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43672e29-42f4-4d88-a8c2-844dbbfd7c1a None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:53:00.012856 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfddec4-7d52-4e57-9841-bfe72f7ee568 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:53:00.013514 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfddec4-7d52-4e57-9841-bfe72f7ee568 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:53:00.013792 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfddec4-7d52-4e57-9841-bfe72f7ee568 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:53:00.014057 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfddec4-7d52-4e57-9841-bfe72f7ee568 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:53:00.016111 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-3bfddec4-7d52-4e57-9841-bfe72f7ee568 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:53:00.089852 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfddec4-7d52-4e57-9841-bfe72f7ee568 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:53:00.091463 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfddec4-7d52-4e57-9841-bfe72f7ee568 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:53:00.101179 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1185/2393] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:52:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:53:18.713398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788859b8-2ea8-4fb0-be2a-776455e8b33d service swift] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:53:18.725872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788859b8-2ea8-4fb0-be2a-776455e8b33d service swift] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:53:18.733184 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04322324-1c0b-4245-affc-4feebff3b852 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:53:18.733816 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04322324-1c0b-4245-affc-4feebff3b852 service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:53:18.734094 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04322324-1c0b-4245-affc-4feebff3b852 service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:53:18.734372 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04322324-1c0b-4245-affc-4feebff3b852 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:53:18.736331 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-04322324-1c0b-4245-affc-4feebff3b852 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:53:18.797601 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04322324-1c0b-4245-affc-4feebff3b852 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:53:18.799266 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04322324-1c0b-4245-affc-4feebff3b852 service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:53:18.808610 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1209/2394] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:53:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:53:34.245508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfddec4-7d52-4e57-9841-bfe72f7ee568 service nova] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:53:34.257105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfddec4-7d52-4e57-9841-bfe72f7ee568 service nova] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:53:34.264074 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea641168-68fb-4ef6-a200-6994a99a7c7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:53:34.264756 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea641168-68fb-4ef6-a200-6994a99a7c7a service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:53:34.265037 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea641168-68fb-4ef6-a200-6994a99a7c7a service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:53:34.265306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea641168-68fb-4ef6-a200-6994a99a7c7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:53:34.274261 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-ea641168-68fb-4ef6-a200-6994a99a7c7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:53:34.328863 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea641168-68fb-4ef6-a200-6994a99a7c7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:53:34.330526 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea641168-68fb-4ef6-a200-6994a99a7c7a service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:53:34.341312 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1186/2395] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:53:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:54:05.647114 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04322324-1c0b-4245-affc-4feebff3b852 service nova] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:54:05.659108 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04322324-1c0b-4245-affc-4feebff3b852 service nova] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:54:05.665917 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ac98b-425e-43aa-b8cb-d36238c207b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:54:05.666845 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ac98b-425e-43aa-b8cb-d36238c207b0 service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:54:05.667127 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ac98b-425e-43aa-b8cb-d36238c207b0 service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:54:05.667396 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ac98b-425e-43aa-b8cb-d36238c207b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:54:05.669366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-009ac98b-425e-43aa-b8cb-d36238c207b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:54:05.682570 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ac98b-425e-43aa-b8cb-d36238c207b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:54:05.684194 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ac98b-425e-43aa-b8cb-d36238c207b0 service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:54:05.722572 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1210/2396] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:54:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:54:36.721558 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cae34e-a7aa-41a5-9704-8cb127f1e857 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:54:36.722157 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cae34e-a7aa-41a5-9704-8cb127f1e857 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:54:36.722157 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cae34e-a7aa-41a5-9704-8cb127f1e857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:54:36.743349 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-64cae34e-a7aa-41a5-9704-8cb127f1e857 None None] MFA Rules not processed for user `e58c14dca7da4e6495f7e08f2bb127e2`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 09:54:36.744597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-64cae34e-a7aa-41a5-9704-8cb127f1e857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:54:36.753371 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1187/2397] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 09:54:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 09:55:26.453386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ac98b-425e-43aa-b8cb-d36238c207b0 service nova] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:55:26.469807 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ac98b-425e-43aa-b8cb-d36238c207b0 service nova] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:55:26.478499 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa3625-007f-4155-97a8-33f39f4d9ca9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9e458c2c5f4ed1b3eddd84c2ec1703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:55:26.479166 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa3625-007f-4155-97a8-33f39f4d9ca9 service placement] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:55:26.479828 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa3625-007f-4155-97a8-33f39f4d9ca9 service placement] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:55:26.479828 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa3625-007f-4155-97a8-33f39f4d9ca9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:55:26.482688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-5ffa3625-007f-4155-97a8-33f39f4d9ca9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:55:26.495123 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa3625-007f-4155-97a8-33f39f4d9ca9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:55:26.496737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa3625-007f-4155-97a8-33f39f4d9ca9 service placement] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:55:26.506730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1211/2398] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:55:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:56:04.277969 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cae34e-a7aa-41a5-9704-8cb127f1e857 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:56:04.279505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-64cae34e-a7aa-41a5-9704-8cb127f1e857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:56:04.338510 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cae34e-a7aa-41a5-9704-8cb127f1e857 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:56:04.342798 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ec769-f245-4a82-b2e3-4d345e2ea238 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:56:04.343426 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ec769-f245-4a82-b2e3-4d345e2ea238 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:56:04.343700 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ec769-f245-4a82-b2e3-4d345e2ea238 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:56:04.343969 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ec769-f245-4a82-b2e3-4d345e2ea238 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:56:04.345994 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a31ec769-f245-4a82-b2e3-4d345e2ea238 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:56:04.357269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ec769-f245-4a82-b2e3-4d345e2ea238 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:56:04.358685 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ec769-f245-4a82-b2e3-4d345e2ea238 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:56:04.367558 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1188/2399] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:56:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:56:04.555612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa3625-007f-4155-97a8-33f39f4d9ca9 service placement] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:56:04.568167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa3625-007f-4155-97a8-33f39f4d9ca9 service placement] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:56:04.573659 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610c79a-84e7-4cf5-8ccb-b951d62afb2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:56:04.574301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610c79a-84e7-4cf5-8ccb-b951d62afb2a service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:56:04.574584 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610c79a-84e7-4cf5-8ccb-b951d62afb2a service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:56:04.574888 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610c79a-84e7-4cf5-8ccb-b951d62afb2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:56:04.576695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b610c79a-84e7-4cf5-8ccb-b951d62afb2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 09:56:04.590056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b610c79a-84e7-4cf5-8ccb-b951d62afb2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:56:04.591817 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b610c79a-84e7-4cf5-8ccb-b951d62afb2a service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:56:04.601041 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1212/2400] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:56:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:58:04.567560 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ec769-f245-4a82-b2e3-4d345e2ea238 service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:58:04.580345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ec769-f245-4a82-b2e3-4d345e2ea238 service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:58:04.586276 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c8901-1cfa-4019-b318-3168cdbc9938 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:58:04.586924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c8901-1cfa-4019-b318-3168cdbc9938 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:58:04.587212 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c8901-1cfa-4019-b318-3168cdbc9938 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:58:04.587626 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c8901-1cfa-4019-b318-3168cdbc9938 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:58:04.600540 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c8901-1cfa-4019-b318-3168cdbc9938 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:58:04.602036 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c8901-1cfa-4019-b318-3168cdbc9938 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:58:04.613172 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1189/2401] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:58:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:58:04.737350 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610c79a-84e7-4cf5-8ccb-b951d62afb2a service nova] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:58:04.747288 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610c79a-84e7-4cf5-8ccb-b951d62afb2a service nova] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:58:04.751635 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a6d40-07eb-4b57-9133-6f76b011a2b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:58:04.752271 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a6d40-07eb-4b57-9133-6f76b011a2b0 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:58:04.752547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a6d40-07eb-4b57-9133-6f76b011a2b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:58:04.752854 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a6d40-07eb-4b57-9133-6f76b011a2b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:58:04.763545 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a6d40-07eb-4b57-9133-6f76b011a2b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:58:04.765021 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a6d40-07eb-4b57-9133-6f76b011a2b0 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:58:04.775897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1213/2402] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:58:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:58:04.884129 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c8901-1cfa-4019-b318-3168cdbc9938 service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:58:04.893568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c8901-1cfa-4019-b318-3168cdbc9938 service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:58:04.897953 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1166520-05dd-477f-84e2-2cb9044e505c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:58:04.898585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1166520-05dd-477f-84e2-2cb9044e505c service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:58:04.898858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1166520-05dd-477f-84e2-2cb9044e505c service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:58:04.899121 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1166520-05dd-477f-84e2-2cb9044e505c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:58:04.908614 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1166520-05dd-477f-84e2-2cb9044e505c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:58:04.910060 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1166520-05dd-477f-84e2-2cb9044e505c service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:58:04.937910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1190/2403] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:58:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:58:06.291513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a6d40-07eb-4b57-9133-6f76b011a2b0 service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:58:06.303371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a6d40-07eb-4b57-9133-6f76b011a2b0 service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:58:06.320113 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f911aeb-98c7-4723-8236-bb69e50de297 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:58:06.320806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f911aeb-98c7-4723-8236-bb69e50de297 service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:58:06.321086 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f911aeb-98c7-4723-8236-bb69e50de297 service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:58:06.321358 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f911aeb-98c7-4723-8236-bb69e50de297 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:58:06.339046 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f911aeb-98c7-4723-8236-bb69e50de297 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:58:06.340647 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f911aeb-98c7-4723-8236-bb69e50de297 service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:58:06.368440 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1214/2404] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:58:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 09:58:28.098105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1166520-05dd-477f-84e2-2cb9044e505c service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 09:58:28.113048 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1166520-05dd-477f-84e2-2cb9044e505c service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 09:58:28.118971 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f981f7-5588-4958-a6da-8ddccfda0581 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 09:58:28.119583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f981f7-5588-4958-a6da-8ddccfda0581 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 09:58:28.119868 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f981f7-5588-4958-a6da-8ddccfda0581 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 09:58:28.120156 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f981f7-5588-4958-a6da-8ddccfda0581 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 09:58:28.128888 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f981f7-5588-4958-a6da-8ddccfda0581 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 09:58:28.130339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f981f7-5588-4958-a6da-8ddccfda0581 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 09:58:28.143379 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1191/2405] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 09:58:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:00:05.100075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f911aeb-98c7-4723-8236-bb69e50de297 service nova] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:00:05.137609 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f911aeb-98c7-4723-8236-bb69e50de297 service nova] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:00:05.141478 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a25a72-6dd5-4cd4-a50f-6d49f167448a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:00:05.142116 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a25a72-6dd5-4cd4-a50f-6d49f167448a service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:00:05.142394 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a25a72-6dd5-4cd4-a50f-6d49f167448a service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:00:05.142664 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a25a72-6dd5-4cd4-a50f-6d49f167448a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:00:05.163051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a25a72-6dd5-4cd4-a50f-6d49f167448a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:00:05.164607 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a25a72-6dd5-4cd4-a50f-6d49f167448a service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:00:05.189671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1215/2406] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:00:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:00:06.465364 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f981f7-5588-4958-a6da-8ddccfda0581 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:00:06.515873 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f981f7-5588-4958-a6da-8ddccfda0581 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:00:06.524138 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0565ae-e26c-4233-9af0-85e2460b6960 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:00:06.524912 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0565ae-e26c-4233-9af0-85e2460b6960 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:00:06.525268 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0565ae-e26c-4233-9af0-85e2460b6960 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:00:06.525612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0565ae-e26c-4233-9af0-85e2460b6960 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:00:06.540752 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a25a72-6dd5-4cd4-a50f-6d49f167448a service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:00:06.542177 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c8a25a72-6dd5-4cd4-a50f-6d49f167448a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:00:06.551888 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0565ae-e26c-4233-9af0-85e2460b6960 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:00:06.553890 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0565ae-e26c-4233-9af0-85e2460b6960 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:00:06.581806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a25a72-6dd5-4cd4-a50f-6d49f167448a service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:00:06.585420 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37eb77-efcf-45c6-a95a-25a69283cc03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:00:06.586061 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37eb77-efcf-45c6-a95a-25a69283cc03 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:00:06.586335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37eb77-efcf-45c6-a95a-25a69283cc03 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:00:06.586605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37eb77-efcf-45c6-a95a-25a69283cc03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:00:06.586942 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1192/2407] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:00:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:00:06.595488 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a37eb77-efcf-45c6-a95a-25a69283cc03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:00:06.596977 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a37eb77-efcf-45c6-a95a-25a69283cc03 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:00:06.606417 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1216/2408] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:00:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 10:00:29.422168 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0565ae-e26c-4233-9af0-85e2460b6960 service nova] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:00:29.423667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-4d0565ae-e26c-4233-9af0-85e2460b6960 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:00:29.462566 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0565ae-e26c-4233-9af0-85e2460b6960 service nova] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:00:29.469046 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c6d58-0799-4258-ac97-a776b0dc6491 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9e458c2c5f4ed1b3eddd84c2ec1703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:00:29.469696 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c6d58-0799-4258-ac97-a776b0dc6491 service placement] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:00:29.469988 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c6d58-0799-4258-ac97-a776b0dc6491 service placement] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:00:29.470252 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c6d58-0799-4258-ac97-a776b0dc6491 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:00:29.479647 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c6d58-0799-4258-ac97-a776b0dc6491 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:00:29.481289 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c6d58-0799-4258-ac97-a776b0dc6491 service placement] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:00:29.491247 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1193/2409] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:00:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:02:05.388897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37eb77-efcf-45c6-a95a-25a69283cc03 service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:02:05.398517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37eb77-efcf-45c6-a95a-25a69283cc03 service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:02:05.402774 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47bf45-5228-435e-abc9-c18d1727e32b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:02:05.403423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47bf45-5228-435e-abc9-c18d1727e32b service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:02:05.403701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47bf45-5228-435e-abc9-c18d1727e32b service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:02:05.403984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47bf45-5228-435e-abc9-c18d1727e32b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:02:05.468950 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f47bf45-5228-435e-abc9-c18d1727e32b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:02:05.470386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f47bf45-5228-435e-abc9-c18d1727e32b service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:02:05.494946 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1217/2410] 10.222.0.26 () {60 vars in 1396 bytes} [Mon May 9 10:02:05 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:02:06.819972 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c6d58-0799-4258-ac97-a776b0dc6491 service placement] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:02:06.902326 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c6d58-0799-4258-ac97-a776b0dc6491 service placement] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:02:06.908233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9adf5-771a-4c42-8827-bbb52522b8f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:02:06.908904 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9adf5-771a-4c42-8827-bbb52522b8f2 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:02:06.909181 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9adf5-771a-4c42-8827-bbb52522b8f2 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:02:06.909451 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9adf5-771a-4c42-8827-bbb52522b8f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:02:06.917478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9adf5-771a-4c42-8827-bbb52522b8f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:02:06.919065 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9adf5-771a-4c42-8827-bbb52522b8f2 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:02:06.945633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1194/2411] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:02:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:03:33.838586 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47bf45-5228-435e-abc9-c18d1727e32b service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:03:33.840389 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-1f47bf45-5228-435e-abc9-c18d1727e32b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:03:33.923869 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47bf45-5228-435e-abc9-c18d1727e32b service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:03:33.928365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb060bb6-74bd-4574-9705-70ad87de1989 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:03:33.929037 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb060bb6-74bd-4574-9705-70ad87de1989 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:03:33.929316 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb060bb6-74bd-4574-9705-70ad87de1989 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:03:33.929590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb060bb6-74bd-4574-9705-70ad87de1989 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:03:33.931563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-eb060bb6-74bd-4574-9705-70ad87de1989 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:03:33.941416 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb060bb6-74bd-4574-9705-70ad87de1989 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:03:33.942852 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb060bb6-74bd-4574-9705-70ad87de1989 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:03:33.950958 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1218/2412] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:03:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:04:05.571282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9adf5-771a-4c42-8827-bbb52522b8f2 service nova] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:04:05.582186 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9adf5-771a-4c42-8827-bbb52522b8f2 service nova] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:04:05.586719 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff423edf-4881-41b8-87dd-5f06733ca12c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:04:05.587360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff423edf-4881-41b8-87dd-5f06733ca12c service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:04:05.587624 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff423edf-4881-41b8-87dd-5f06733ca12c service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:04:05.587884 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff423edf-4881-41b8-87dd-5f06733ca12c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:04:05.590018 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-ff423edf-4881-41b8-87dd-5f06733ca12c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:04:05.621208 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff423edf-4881-41b8-87dd-5f06733ca12c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:04:05.622677 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff423edf-4881-41b8-87dd-5f06733ca12c service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:04:05.634287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1195/2413] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:04:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:04:05.642122 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb060bb6-74bd-4574-9705-70ad87de1989 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:04:05.652245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb060bb6-74bd-4574-9705-70ad87de1989 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:04:05.656699 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae15b92-886b-4f26-ae19-662621440fb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:04:05.657365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae15b92-886b-4f26-ae19-662621440fb7 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:04:05.657640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae15b92-886b-4f26-ae19-662621440fb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:04:05.657938 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae15b92-886b-4f26-ae19-662621440fb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:04:05.660077 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-0ae15b92-886b-4f26-ae19-662621440fb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:04:05.746080 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae15b92-886b-4f26-ae19-662621440fb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:04:05.748054 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae15b92-886b-4f26-ae19-662621440fb7 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:04:05.758714 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1219/2414] 10.222.0.26 () {60 vars in 1396 bytes} [Mon May 9 10:04:05 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:04:05.811727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff423edf-4881-41b8-87dd-5f06733ca12c service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:04:05.820518 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff423edf-4881-41b8-87dd-5f06733ca12c service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:04:05.824863 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47275a-7a83-4365-801f-04ed31984ba7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:04:05.825647 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47275a-7a83-4365-801f-04ed31984ba7 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:04:05.826013 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47275a-7a83-4365-801f-04ed31984ba7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:04:05.826376 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47275a-7a83-4365-801f-04ed31984ba7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:04:05.828398 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-9c47275a-7a83-4365-801f-04ed31984ba7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:04:05.843968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47275a-7a83-4365-801f-04ed31984ba7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:04:05.845432 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47275a-7a83-4365-801f-04ed31984ba7 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:04:05.857109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1196/2415] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:04:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:04:06.121080 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae15b92-886b-4f26-ae19-662621440fb7 service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:04:06.121080 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-0ae15b92-886b-4f26-ae19-662621440fb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:04:06.136493 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae15b92-886b-4f26-ae19-662621440fb7 service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:04:06.144514 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bcf6a-5732-42bd-975e-a1bba359ee5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:04:06.145405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bcf6a-5732-42bd-975e-a1bba359ee5e service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:04:06.145678 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bcf6a-5732-42bd-975e-a1bba359ee5e service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:04:06.145944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bcf6a-5732-42bd-975e-a1bba359ee5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:04:06.148029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-119bcf6a-5732-42bd-975e-a1bba359ee5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:04:06.242375 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bcf6a-5732-42bd-975e-a1bba359ee5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:04:06.244096 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bcf6a-5732-42bd-975e-a1bba359ee5e service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:04:06.255235 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1220/2416] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:04:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:04:07.614849 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ab8a3-2977-40c1-8960-08bb6438a2bf None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:04:07.615328 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ab8a3-2977-40c1-8960-08bb6438a2bf None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:04:07.615328 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ab8a3-2977-40c1-8960-08bb6438a2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:04:07.671672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-6b4ab8a3-2977-40c1-8960-08bb6438a2bf None None] MFA Rules not processed for user `1c091a71e2ad46ceafe25be874c016a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:04:07.671672 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-6b4ab8a3-2977-40c1-8960-08bb6438a2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:04:07.731443 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1197/2417] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:04:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:04:08.292218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bcf6a-5732-42bd-975e-a1bba359ee5e service nova] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:04:08.302027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bcf6a-5732-42bd-975e-a1bba359ee5e service nova] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:04:08.306566 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1b010-51a5-40a4-bbbf-d7907d1a995d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:04:08.307210 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1b010-51a5-40a4-bbbf-d7907d1a995d service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:04:08.307501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1b010-51a5-40a4-bbbf-d7907d1a995d service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:04:08.307741 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1b010-51a5-40a4-bbbf-d7907d1a995d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:04:08.309951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-97c1b010-51a5-40a4-bbbf-d7907d1a995d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:04:08.322643 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1b010-51a5-40a4-bbbf-d7907d1a995d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:04:08.325081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1b010-51a5-40a4-bbbf-d7907d1a995d service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:04:08.332237 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1221/2418] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:04:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:05:03.840692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ab8a3-2977-40c1-8960-08bb6438a2bf None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:05:03.842352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-6b4ab8a3-2977-40c1-8960-08bb6438a2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:05:03.859901 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ab8a3-2977-40c1-8960-08bb6438a2bf None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:05:03.866740 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb8186-3331-4299-8f79-cd806593be32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:05:03.867390 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb8186-3331-4299-8f79-cd806593be32 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:05:03.867666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb8186-3331-4299-8f79-cd806593be32 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:05:03.867979 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb8186-3331-4299-8f79-cd806593be32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:05:03.869967 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-8ccb8186-3331-4299-8f79-cd806593be32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:05:03.939608 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb8186-3331-4299-8f79-cd806593be32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:05:03.941498 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb8186-3331-4299-8f79-cd806593be32 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:05:03.951497 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1198/2419] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:05:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:05:07.444642 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1b010-51a5-40a4-bbbf-d7907d1a995d service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:05:07.454793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1b010-51a5-40a4-bbbf-d7907d1a995d service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:05:07.459585 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d364b-bbe8-4040-8777-82f87a4f22c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:05:07.460204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d364b-bbe8-4040-8777-82f87a4f22c3 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:05:07.460512 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d364b-bbe8-4040-8777-82f87a4f22c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:05:07.460816 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d364b-bbe8-4040-8777-82f87a4f22c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:05:07.463019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-932d364b-bbe8-4040-8777-82f87a4f22c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:05:07.475790 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d364b-bbe8-4040-8777-82f87a4f22c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:05:07.477276 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d364b-bbe8-4040-8777-82f87a4f22c3 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:05:07.487604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1222/2420] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:05:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:05:32.469059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb8186-3331-4299-8f79-cd806593be32 service nova] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:05:32.536313 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb8186-3331-4299-8f79-cd806593be32 service nova] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:05:32.541333 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa94b49-663e-4a12-8563-17b5610fe8ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9e458c2c5f4ed1b3eddd84c2ec1703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:05:32.541984 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa94b49-663e-4a12-8563-17b5610fe8ff service placement] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:05:32.542261 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa94b49-663e-4a12-8563-17b5610fe8ff service placement] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:05:32.542522 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa94b49-663e-4a12-8563-17b5610fe8ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:05:32.544466 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-9aa94b49-663e-4a12-8563-17b5610fe8ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:05:32.556168 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa94b49-663e-4a12-8563-17b5610fe8ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:05:32.557819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa94b49-663e-4a12-8563-17b5610fe8ff service placement] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:05:32.568147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1199/2421] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:05:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 10:07:05.733235 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d364b-bbe8-4040-8777-82f87a4f22c3 service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:07:05.735213 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-932d364b-bbe8-4040-8777-82f87a4f22c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:07:05.794420 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d364b-bbe8-4040-8777-82f87a4f22c3 service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:07:05.799469 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb587da-3afa-4a70-b6ea-07ae0d5de447 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:07:05.800154 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb587da-3afa-4a70-b6ea-07ae0d5de447 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:07:05.800433 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb587da-3afa-4a70-b6ea-07ae0d5de447 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:07:05.800730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb587da-3afa-4a70-b6ea-07ae0d5de447 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:07:05.802847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-8fb587da-3afa-4a70-b6ea-07ae0d5de447 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:07:05.827565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb587da-3afa-4a70-b6ea-07ae0d5de447 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:07:05.829322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb587da-3afa-4a70-b6ea-07ae0d5de447 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:07:05.838545 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1223/2422] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:07:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:07:06.406019 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa94b49-663e-4a12-8563-17b5610fe8ff service placement] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:07:06.416001 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa94b49-663e-4a12-8563-17b5610fe8ff service placement] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:07:06.420577 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6fd5d-3f86-4e0f-8e49-3a643019abfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:07:06.421255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6fd5d-3f86-4e0f-8e49-3a643019abfd service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:07:06.421530 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6fd5d-3f86-4e0f-8e49-3a643019abfd service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:07:06.421914 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6fd5d-3f86-4e0f-8e49-3a643019abfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:07:06.424067 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-52c6fd5d-3f86-4e0f-8e49-3a643019abfd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:07:06.504328 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6fd5d-3f86-4e0f-8e49-3a643019abfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:07:06.505806 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6fd5d-3f86-4e0f-8e49-3a643019abfd service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:07:06.515017 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1200/2423] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:07:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:08:37.077778 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb587da-3afa-4a70-b6ea-07ae0d5de447 service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:08:37.088024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb587da-3afa-4a70-b6ea-07ae0d5de447 service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:08:37.092537 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c33019-78d8-4c67-9724-e90fac9000a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:08:37.093476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c33019-78d8-4c67-9724-e90fac9000a3 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:08:37.093817 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c33019-78d8-4c67-9724-e90fac9000a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:08:37.094095 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c33019-78d8-4c67-9724-e90fac9000a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:08:37.102300 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c33019-78d8-4c67-9724-e90fac9000a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:08:37.103723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c33019-78d8-4c67-9724-e90fac9000a3 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:08:37.112087 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1224/2424] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:08:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:09:05.256877 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6fd5d-3f86-4e0f-8e49-3a643019abfd service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:09:05.270596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6fd5d-3f86-4e0f-8e49-3a643019abfd service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:09:05.277693 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215c265-514b-4582-9ea8-fa6873415ce6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:09:05.278339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215c265-514b-4582-9ea8-fa6873415ce6 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:09:05.278612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215c265-514b-4582-9ea8-fa6873415ce6 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:09:05.278884 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215c265-514b-4582-9ea8-fa6873415ce6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:09:05.287463 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d215c265-514b-4582-9ea8-fa6873415ce6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:09:05.289145 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d215c265-514b-4582-9ea8-fa6873415ce6 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:09:05.317990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1201/2425] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:09:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:09:05.546275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c33019-78d8-4c67-9724-e90fac9000a3 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:09:05.557014 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c33019-78d8-4c67-9724-e90fac9000a3 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:09:05.561717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387870ea-dd0f-4805-bfaf-538a9b6e435b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:09:05.562361 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387870ea-dd0f-4805-bfaf-538a9b6e435b service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:09:05.562639 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387870ea-dd0f-4805-bfaf-538a9b6e435b service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:09:05.562907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387870ea-dd0f-4805-bfaf-538a9b6e435b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:09:05.572688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387870ea-dd0f-4805-bfaf-538a9b6e435b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:09:05.574127 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387870ea-dd0f-4805-bfaf-538a9b6e435b service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:09:05.602337 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1225/2426] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:09:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:10:04.820054 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215c265-514b-4582-9ea8-fa6873415ce6 service nova] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:10:04.830182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215c265-514b-4582-9ea8-fa6873415ce6 service nova] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:10:04.834658 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395dae44-30fd-4cc7-91ae-afadc7a08a43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:10:04.835329 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395dae44-30fd-4cc7-91ae-afadc7a08a43 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:10:04.835608 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395dae44-30fd-4cc7-91ae-afadc7a08a43 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:10:04.835880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395dae44-30fd-4cc7-91ae-afadc7a08a43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:10:04.845520 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395dae44-30fd-4cc7-91ae-afadc7a08a43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:10:04.846965 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395dae44-30fd-4cc7-91ae-afadc7a08a43 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:10:04.856973 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1202/2427] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:10:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:10:07.548341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387870ea-dd0f-4805-bfaf-538a9b6e435b service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:10:07.594593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387870ea-dd0f-4805-bfaf-538a9b6e435b service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:10:07.599214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c599c98-84e6-4b2b-8281-e141c4cc487c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:10:07.599864 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c599c98-84e6-4b2b-8281-e141c4cc487c service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:10:07.600140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c599c98-84e6-4b2b-8281-e141c4cc487c service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:10:07.600412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c599c98-84e6-4b2b-8281-e141c4cc487c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:10:07.623599 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c599c98-84e6-4b2b-8281-e141c4cc487c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:10:07.625169 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c599c98-84e6-4b2b-8281-e141c4cc487c service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:10:07.652512 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1226/2428] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:10:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:10:08.179244 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395dae44-30fd-4cc7-91ae-afadc7a08a43 service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:10:08.200522 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395dae44-30fd-4cc7-91ae-afadc7a08a43 service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:10:08.205655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ffce7-6262-4410-9f5e-e7f352488e8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:10:08.206314 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ffce7-6262-4410-9f5e-e7f352488e8f service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:10:08.206588 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ffce7-6262-4410-9f5e-e7f352488e8f service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:10:08.206886 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ffce7-6262-4410-9f5e-e7f352488e8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:10:08.227613 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ffce7-6262-4410-9f5e-e7f352488e8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:10:08.229154 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ffce7-6262-4410-9f5e-e7f352488e8f service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:10:08.240188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1203/2429] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:10:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:10:09.440163 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c599c98-84e6-4b2b-8281-e141c4cc487c service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:10:09.441915 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-5c599c98-84e6-4b2b-8281-e141c4cc487c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:10:09.470445 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c599c98-84e6-4b2b-8281-e141c4cc487c service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:10:09.474784 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb8047-1b51-49f8-b7f0-058367924e09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:10:09.475718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb8047-1b51-49f8-b7f0-058367924e09 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:10:09.476345 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb8047-1b51-49f8-b7f0-058367924e09 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:10:09.476612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb8047-1b51-49f8-b7f0-058367924e09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:10:09.491399 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb8047-1b51-49f8-b7f0-058367924e09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:10:09.493101 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb8047-1b51-49f8-b7f0-058367924e09 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:10:09.505998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1227/2430] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:10:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:10:10.168543 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ffce7-6262-4410-9f5e-e7f352488e8f service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:10:10.177857 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ffce7-6262-4410-9f5e-e7f352488e8f service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:10:10.182141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dda9a4-df40-49dd-8600-d1e9b00cb955 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:10:10.182805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dda9a4-df40-49dd-8600-d1e9b00cb955 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:10:10.183080 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dda9a4-df40-49dd-8600-d1e9b00cb955 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:10:10.183409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dda9a4-df40-49dd-8600-d1e9b00cb955 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:10:10.185531 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-29dda9a4-df40-49dd-8600-d1e9b00cb955 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:10:10.276263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dda9a4-df40-49dd-8600-d1e9b00cb955 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:10:10.278414 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dda9a4-df40-49dd-8600-d1e9b00cb955 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:10:10.298844 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1204/2431] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:10:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:10:17.993578 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb8047-1b51-49f8-b7f0-058367924e09 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:10:17.995195 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-fddb8047-1b51-49f8-b7f0-058367924e09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:10:18.011862 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb8047-1b51-49f8-b7f0-058367924e09 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:10:18.017318 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f547d6-5382-437b-8259-a0a56a856272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:10:18.018254 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f547d6-5382-437b-8259-a0a56a856272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:10:18.018559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f547d6-5382-437b-8259-a0a56a856272 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:10:18.018824 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f547d6-5382-437b-8259-a0a56a856272 admin admin] PATH_INFO: `/v3/users/1c091a71e2ad46ceafe25be874c016a6` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:10:18.020092 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f547d6-5382-437b-8259-a0a56a856272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c091a71e2ad46ceafe25be874c016a6)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:10:18.022215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f547d6-5382-437b-8259-a0a56a856272 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:10:18.068250 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-70f547d6-5382-437b-8259-a0a56a856272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c091a71e2ad46ceafe25be874c016a6'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:10:18.076808 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-70f547d6-5382-437b-8259-a0a56a856272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c091a71e2ad46ceafe25be874c016a6'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:10:18.076808 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-70f547d6-5382-437b-8259-a0a56a856272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c091a71e2ad46ceafe25be874c016a6'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:10:18.087836 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-70f547d6-5382-437b-8259-a0a56a856272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c091a71e2ad46ceafe25be874c016a6'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:10:18.088486 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-70f547d6-5382-437b-8259-a0a56a856272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c091a71e2ad46ceafe25be874c016a6'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:10:18.106613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1228/2432] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:10:17 2022] DELETE /identity/v3/users/1c091a71e2ad46ceafe25be874c016a6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 10:10:18.338345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dda9a4-df40-49dd-8600-d1e9b00cb955 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:10:18.349030 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dda9a4-df40-49dd-8600-d1e9b00cb955 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:10:18.406522 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081efc8-8f54-4d43-add7-f727d8c386b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:10:18.407389 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081efc8-8f54-4d43-add7-f727d8c386b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:10:18.407662 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081efc8-8f54-4d43-add7-f727d8c386b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:10:18.407952 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081efc8-8f54-4d43-add7-f727d8c386b6 admin admin] PATH_INFO: `/v3/projects/39c36cb22c2846e480c26840635bf29c` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:10:18.409330 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081efc8-8f54-4d43-add7-f727d8c386b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39c36cb22c2846e480c26840635bf29c)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:10:18.411231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081efc8-8f54-4d43-add7-f727d8c386b6 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:10:18.459206 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-9081efc8-8f54-4d43-add7-f727d8c386b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39c36cb22c2846e480c26840635bf29c', 'request_id': 'req-9081efc8-8f54-4d43-add7-f727d8c386b6'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:10:18.465784 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-9081efc8-8f54-4d43-add7-f727d8c386b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39c36cb22c2846e480c26840635bf29c', 'request_id': 'req-9081efc8-8f54-4d43-add7-f727d8c386b6'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:10:18.474723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-9081efc8-8f54-4d43-add7-f727d8c386b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39c36cb22c2846e480c26840635bf29c', 'request_id': 'req-9081efc8-8f54-4d43-add7-f727d8c386b6'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:10:18.487730 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-9081efc8-8f54-4d43-add7-f727d8c386b6 admin admin] The token cache is being invalidate because project 39c36cb22c2846e480c26840635bf29c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:10:18.488010 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-9081efc8-8f54-4d43-add7-f727d8c386b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:10:18.488613 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-9081efc8-8f54-4d43-add7-f727d8c386b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:10:18.490269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1205/2433] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:10:18 2022] DELETE /identity/v3/projects/39c36cb22c2846e480c26840635bf29c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:10:18.494350 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f547d6-5382-437b-8259-a0a56a856272 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:10:18.495985 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-70f547d6-5382-437b-8259-a0a56a856272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:10:18.509424 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f547d6-5382-437b-8259-a0a56a856272 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:10:18.575229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560bca8-f539-47fe-ad33-6524d6bf5481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:10:18.576122 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560bca8-f539-47fe-ad33-6524d6bf5481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:10:18.576403 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560bca8-f539-47fe-ad33-6524d6bf5481 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:10:18.576694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560bca8-f539-47fe-ad33-6524d6bf5481 admin admin] PATH_INFO: `/v3/users/e536207e603644b69ad040b27970b773` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:10:18.590673 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560bca8-f539-47fe-ad33-6524d6bf5481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e536207e603644b69ad040b27970b773)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:10:18.592610 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560bca8-f539-47fe-ad33-6524d6bf5481 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:10:18.643210 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-b560bca8-f539-47fe-ad33-6524d6bf5481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e536207e603644b69ad040b27970b773'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:10:18.650312 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-b560bca8-f539-47fe-ad33-6524d6bf5481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e536207e603644b69ad040b27970b773'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:10:18.651068 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-b560bca8-f539-47fe-ad33-6524d6bf5481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e536207e603644b69ad040b27970b773'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:10:18.659794 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-b560bca8-f539-47fe-ad33-6524d6bf5481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e536207e603644b69ad040b27970b773'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:10:18.660477 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-b560bca8-f539-47fe-ad33-6524d6bf5481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e536207e603644b69ad040b27970b773'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:10:18.677056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1229/2434] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:10:18 2022] DELETE /identity/v3/users/e536207e603644b69ad040b27970b773 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:10:18.871407 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081efc8-8f54-4d43-add7-f727d8c386b6 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:10:18.872927 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-9081efc8-8f54-4d43-add7-f727d8c386b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:10:18.887304 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081efc8-8f54-4d43-add7-f727d8c386b6 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:10:18.958410 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada8bf5-06c5-4f3b-90f7-ec2145db08db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:10:18.959277 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada8bf5-06c5-4f3b-90f7-ec2145db08db admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:10:18.959554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada8bf5-06c5-4f3b-90f7-ec2145db08db admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:10:18.959821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada8bf5-06c5-4f3b-90f7-ec2145db08db admin admin] PATH_INFO: `/v3/projects/53e4a501080e4803a7f4e7b2a2cf65ee` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:10:18.970242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada8bf5-06c5-4f3b-90f7-ec2145db08db admin admin] RBAC: Authorizing `identity:delete_project(project_id=53e4a501080e4803a7f4e7b2a2cf65ee)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:10:18.972161 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada8bf5-06c5-4f3b-90f7-ec2145db08db admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:10:19.017433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-dada8bf5-06c5-4f3b-90f7-ec2145db08db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53e4a501080e4803a7f4e7b2a2cf65ee', 'request_id': 'req-dada8bf5-06c5-4f3b-90f7-ec2145db08db'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:10:19.024182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-dada8bf5-06c5-4f3b-90f7-ec2145db08db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53e4a501080e4803a7f4e7b2a2cf65ee', 'request_id': 'req-dada8bf5-06c5-4f3b-90f7-ec2145db08db'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:10:19.031003 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-dada8bf5-06c5-4f3b-90f7-ec2145db08db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53e4a501080e4803a7f4e7b2a2cf65ee', 'request_id': 'req-dada8bf5-06c5-4f3b-90f7-ec2145db08db'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:10:19.044915 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-dada8bf5-06c5-4f3b-90f7-ec2145db08db admin admin] The token cache is being invalidate because project 53e4a501080e4803a7f4e7b2a2cf65ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:10:19.045199 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-dada8bf5-06c5-4f3b-90f7-ec2145db08db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:10:19.045825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-dada8bf5-06c5-4f3b-90f7-ec2145db08db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:10:19.047370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1206/2435] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:10:18 2022] DELETE /identity/v3/projects/53e4a501080e4803a7f4e7b2a2cf65ee => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:10:40.123450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560bca8-f539-47fe-ad33-6524d6bf5481 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:10:40.124992 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b560bca8-f539-47fe-ad33-6524d6bf5481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:10:40.219785 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560bca8-f539-47fe-ad33-6524d6bf5481 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:10:40.285309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a55db-53e6-4517-81ff-84f7d1328f2b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9e458c2c5f4ed1b3eddd84c2ec1703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:10:40.285963 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a55db-53e6-4517-81ff-84f7d1328f2b service placement] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:10:40.286223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a55db-53e6-4517-81ff-84f7d1328f2b service placement] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:10:40.286488 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a55db-53e6-4517-81ff-84f7d1328f2b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:10:40.288563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c95a55db-53e6-4517-81ff-84f7d1328f2b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:10:40.300594 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a55db-53e6-4517-81ff-84f7d1328f2b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:10:40.302200 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a55db-53e6-4517-81ff-84f7d1328f2b service placement] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:10:40.312219 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1230/2436] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:10:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:12:06.032452 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada8bf5-06c5-4f3b-90f7-ec2145db08db admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:12:06.034192 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-dada8bf5-06c5-4f3b-90f7-ec2145db08db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:12:06.110860 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada8bf5-06c5-4f3b-90f7-ec2145db08db admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:12:06.116273 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a306ad-553a-4901-a681-0291b171ae61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:12:06.116944 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a306ad-553a-4901-a681-0291b171ae61 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:12:06.117214 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a306ad-553a-4901-a681-0291b171ae61 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:12:06.117478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a306ad-553a-4901-a681-0291b171ae61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:12:06.119253 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a0a306ad-553a-4901-a681-0291b171ae61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:12:06.187238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a306ad-553a-4901-a681-0291b171ae61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:12:06.188826 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a306ad-553a-4901-a681-0291b171ae61 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:12:06.209267 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1207/2437] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:12:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:12:06.478606 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a55db-53e6-4517-81ff-84f7d1328f2b service placement] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:12:06.480182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c95a55db-53e6-4517-81ff-84f7d1328f2b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:12:06.495807 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a55db-53e6-4517-81ff-84f7d1328f2b service placement] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:12:06.502960 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a63b9b-0f4b-418c-8317-a38e0b87ea1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:12:06.503854 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a63b9b-0f4b-418c-8317-a38e0b87ea1e service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:12:06.504122 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a63b9b-0f4b-418c-8317-a38e0b87ea1e service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:12:06.504390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a63b9b-0f4b-418c-8317-a38e0b87ea1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:12:06.506465 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-72a63b9b-0f4b-418c-8317-a38e0b87ea1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:12:06.565966 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a63b9b-0f4b-418c-8317-a38e0b87ea1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:12:06.567597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a63b9b-0f4b-418c-8317-a38e0b87ea1e service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:12:06.576371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1231/2438] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:12:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:12:08.902326 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a306ad-553a-4901-a681-0291b171ae61 service nova] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:12:08.937901 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a306ad-553a-4901-a681-0291b171ae61 service nova] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:12:08.944598 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548961f7-166d-434d-a014-03fe96672c84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:12:08.945409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548961f7-166d-434d-a014-03fe96672c84 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:12:08.945836 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548961f7-166d-434d-a014-03fe96672c84 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:12:08.946081 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548961f7-166d-434d-a014-03fe96672c84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:12:08.948421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-548961f7-166d-434d-a014-03fe96672c84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:12:08.972160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548961f7-166d-434d-a014-03fe96672c84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:12:08.973967 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548961f7-166d-434d-a014-03fe96672c84 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:12:09.007809 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1208/2439] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:12:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:13:41.096096 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a63b9b-0f4b-418c-8317-a38e0b87ea1e service nova] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:13:41.098193 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-72a63b9b-0f4b-418c-8317-a38e0b87ea1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:13:41.179098 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a63b9b-0f4b-418c-8317-a38e0b87ea1e service nova] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:13:41.182917 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffba50-cb9c-481a-b6d6-1a0b26ca7bf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:13:41.183560 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffba50-cb9c-481a-b6d6-1a0b26ca7bf8 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:13:41.183834 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffba50-cb9c-481a-b6d6-1a0b26ca7bf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:13:41.184098 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffba50-cb9c-481a-b6d6-1a0b26ca7bf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:13:41.185965 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-81ffba50-cb9c-481a-b6d6-1a0b26ca7bf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:13:41.196728 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ffba50-cb9c-481a-b6d6-1a0b26ca7bf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:13:41.198483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ffba50-cb9c-481a-b6d6-1a0b26ca7bf8 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:13:41.207135 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1232/2440] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:13:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:14:05.697141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548961f7-166d-434d-a014-03fe96672c84 service nova] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:14:05.709445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548961f7-166d-434d-a014-03fe96672c84 service nova] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:14:05.716563 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9783bc6-9a4c-4dd5-a640-25b1387e18a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:14:05.717252 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9783bc6-9a4c-4dd5-a640-25b1387e18a4 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:14:05.717527 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9783bc6-9a4c-4dd5-a640-25b1387e18a4 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:14:05.717791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9783bc6-9a4c-4dd5-a640-25b1387e18a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:14:05.719900 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-d9783bc6-9a4c-4dd5-a640-25b1387e18a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:14:05.780844 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9783bc6-9a4c-4dd5-a640-25b1387e18a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:14:05.782509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9783bc6-9a4c-4dd5-a640-25b1387e18a4 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:14:05.792600 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1209/2441] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:14:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:14:06.701852 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffba50-cb9c-481a-b6d6-1a0b26ca7bf8 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:14:06.703515 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-81ffba50-cb9c-481a-b6d6-1a0b26ca7bf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:14:06.808071 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffba50-cb9c-481a-b6d6-1a0b26ca7bf8 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:14:06.812473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b57b07-7727-4191-b133-8d14e01306c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:14:06.813170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b57b07-7727-4191-b133-8d14e01306c5 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:14:06.813493 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b57b07-7727-4191-b133-8d14e01306c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:14:06.813786 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b57b07-7727-4191-b133-8d14e01306c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:14:06.815804 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-06b57b07-7727-4191-b133-8d14e01306c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:14:06.892552 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b57b07-7727-4191-b133-8d14e01306c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:14:06.894101 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b57b07-7727-4191-b133-8d14e01306c5 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:14:06.904273 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1233/2442] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:14:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:15:04.750089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9783bc6-9a4c-4dd5-a640-25b1387e18a4 service nova] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:15:04.762220 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9783bc6-9a4c-4dd5-a640-25b1387e18a4 service nova] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:15:04.769484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08c036-3031-4978-ac45-1a290d41809d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:15:04.770158 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08c036-3031-4978-ac45-1a290d41809d service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:15:04.770435 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08c036-3031-4978-ac45-1a290d41809d service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:15:04.770704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08c036-3031-4978-ac45-1a290d41809d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:15:04.772970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-2c08c036-3031-4978-ac45-1a290d41809d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:15:04.859713 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c08c036-3031-4978-ac45-1a290d41809d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:15:04.861492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c08c036-3031-4978-ac45-1a290d41809d service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:15:04.885938 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1210/2443] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:15:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:15:08.622982 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b57b07-7727-4191-b133-8d14e01306c5 service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:15:08.633634 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b57b07-7727-4191-b133-8d14e01306c5 service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:15:08.637727 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15964b3-d126-4858-b9de-8a79dd177d92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:15:08.638401 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15964b3-d126-4858-b9de-8a79dd177d92 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:15:08.638682 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15964b3-d126-4858-b9de-8a79dd177d92 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:15:08.638964 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15964b3-d126-4858-b9de-8a79dd177d92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:15:08.641156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c15964b3-d126-4858-b9de-8a79dd177d92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:15:08.670657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15964b3-d126-4858-b9de-8a79dd177d92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:15:08.672160 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15964b3-d126-4858-b9de-8a79dd177d92 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:15:08.684686 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1234/2444] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:15:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:15:09.371307 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08c036-3031-4978-ac45-1a290d41809d service nova] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:15:09.373567 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-2c08c036-3031-4978-ac45-1a290d41809d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:15:09.393753 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08c036-3031-4978-ac45-1a290d41809d service nova] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:15:09.399660 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58c1c8-ab31-4e54-bddd-2f005f6d3ece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:15:09.400754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58c1c8-ab31-4e54-bddd-2f005f6d3ece service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:15:09.401226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58c1c8-ab31-4e54-bddd-2f005f6d3ece service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:15:09.401713 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58c1c8-ab31-4e54-bddd-2f005f6d3ece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:15:09.404407 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-6d58c1c8-ab31-4e54-bddd-2f005f6d3ece service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:15:09.419617 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58c1c8-ab31-4e54-bddd-2f005f6d3ece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:15:09.421294 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58c1c8-ab31-4e54-bddd-2f005f6d3ece service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:15:09.435125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1211/2445] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:15:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:15:45.402954 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15964b3-d126-4858-b9de-8a79dd177d92 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:15:45.404732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c15964b3-d126-4858-b9de-8a79dd177d92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:15:45.419979 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15964b3-d126-4858-b9de-8a79dd177d92 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:15:45.426197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0791dc-3299-4393-b058-4f4e3aa20e67 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9e458c2c5f4ed1b3eddd84c2ec1703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:15:45.426879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0791dc-3299-4393-b058-4f4e3aa20e67 service placement] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:15:45.427151 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0791dc-3299-4393-b058-4f4e3aa20e67 service placement] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:15:45.427431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0791dc-3299-4393-b058-4f4e3aa20e67 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:15:45.437880 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0791dc-3299-4393-b058-4f4e3aa20e67 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:15:45.439536 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0791dc-3299-4393-b058-4f4e3aa20e67 service placement] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:15:45.451380 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1235/2446] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:15:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:16:08.262502 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d143019-7d08-49e7-bc35-125c31d22c10 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:16:08.263082 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d143019-7d08-49e7-bc35-125c31d22c10 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:16:08.263243 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d143019-7d08-49e7-bc35-125c31d22c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:16:08.329845 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-9d143019-7d08-49e7-bc35-125c31d22c10 None None] MFA Rules not processed for user `af62b2d3f73f482b8db4173af33752c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:16:08.331055 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-9d143019-7d08-49e7-bc35-125c31d22c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:16:08.398749 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1212/2447] 10.222.0.26 () {58 vars in 967 bytes} [Mon May 9 10:16:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:16:08.455576 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0791dc-3299-4393-b058-4f4e3aa20e67 service placement] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:16:08.458592 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-db0791dc-3299-4393-b058-4f4e3aa20e67 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:16:08.550334 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0791dc-3299-4393-b058-4f4e3aa20e67 service placement] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:16:08.555868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32df70c-8700-448a-97dd-77946b09c7f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380e04358f2349aa912d6a97518877cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:16:08.556512 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32df70c-8700-448a-97dd-77946b09c7f7 service swift] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:16:08.556825 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32df70c-8700-448a-97dd-77946b09c7f7 service swift] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:16:08.557400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32df70c-8700-448a-97dd-77946b09c7f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:16:08.567393 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32df70c-8700-448a-97dd-77946b09c7f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:16:08.569036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32df70c-8700-448a-97dd-77946b09c7f7 service swift] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:16:08.577257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1236/2448] 10.222.0.26 () {60 vars in 1384 bytes} [Mon May 9 10:16:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 10:16:09.106669 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d143019-7d08-49e7-bc35-125c31d22c10 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:16:09.118423 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d143019-7d08-49e7-bc35-125c31d22c10 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:16:09.123281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c0a429-77cb-4c21-b01e-648948758086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:16:09.124287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0a429-77cb-4c21-b01e-648948758086 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:16:09.124858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0a429-77cb-4c21-b01e-648948758086 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:16:09.125790 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0a429-77cb-4c21-b01e-648948758086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:16:09.127807 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-07c0a429-77cb-4c21-b01e-648948758086 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:16:09.198497 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c0a429-77cb-4c21-b01e-648948758086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:16:09.199875 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c0a429-77cb-4c21-b01e-648948758086 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:16:09.213183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1213/2449] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:16:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 10:16:12.132760 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14a0c7-06cf-4229-98c4-3ef62ca795cc None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:16:12.134634 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14a0c7-06cf-4229-98c4-3ef62ca795cc None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:16:12.134634 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14a0c7-06cf-4229-98c4-3ef62ca795cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:16:12.192874 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-4a14a0c7-06cf-4229-98c4-3ef62ca795cc None None] MFA Rules not processed for user `76dc34045544496ab030864c5a0ffbff`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:16:12.193239 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-4a14a0c7-06cf-4229-98c4-3ef62ca795cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:16:12.203567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1237/2450] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:16:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:16:12.545732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c0a429-77cb-4c21-b01e-648948758086 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:16:12.554977 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c0a429-77cb-4c21-b01e-648948758086 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:16:12.559747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053900f9-ccce-47fa-9c3a-96a2b36af396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:16:12.560445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053900f9-ccce-47fa-9c3a-96a2b36af396 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:16:12.560755 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053900f9-ccce-47fa-9c3a-96a2b36af396 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:16:12.561559 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053900f9-ccce-47fa-9c3a-96a2b36af396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:16:12.563200 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-053900f9-ccce-47fa-9c3a-96a2b36af396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:16:12.574273 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053900f9-ccce-47fa-9c3a-96a2b36af396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:16:12.575717 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053900f9-ccce-47fa-9c3a-96a2b36af396 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:16:12.584269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1214/2451] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:16:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:16:12.938659 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14a0c7-06cf-4229-98c4-3ef62ca795cc None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:16:12.940136 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-4a14a0c7-06cf-4229-98c4-3ef62ca795cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:16:12.955003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14a0c7-06cf-4229-98c4-3ef62ca795cc None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:16:12.960074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6048dd-f224-4c96-a93c-b100b993e9ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9e458c2c5f4ed1b3eddd84c2ec1703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:16:12.960754 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6048dd-f224-4c96-a93c-b100b993e9ad service placement] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:16:12.961030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6048dd-f224-4c96-a93c-b100b993e9ad service placement] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:16:12.961299 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6048dd-f224-4c96-a93c-b100b993e9ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:16:12.963032 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-5f6048dd-f224-4c96-a93c-b100b993e9ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:16:12.975528 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6048dd-f224-4c96-a93c-b100b993e9ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:16:12.977414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6048dd-f224-4c96-a93c-b100b993e9ad service placement] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:16:12.988869 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1238/2452] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:16:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 10:16:21.584837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053900f9-ccce-47fa-9c3a-96a2b36af396 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:16:21.598937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053900f9-ccce-47fa-9c3a-96a2b36af396 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:16:21.604474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d236a-ed06-44ae-9356-31efc9cf7760 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:16:21.605314 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d236a-ed06-44ae-9356-31efc9cf7760 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:16:21.605594 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d236a-ed06-44ae-9356-31efc9cf7760 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:16:21.605864 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d236a-ed06-44ae-9356-31efc9cf7760 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:16:21.607657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-095d236a-ed06-44ae-9356-31efc9cf7760 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:16:21.619876 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d236a-ed06-44ae-9356-31efc9cf7760 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:16:21.621695 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d236a-ed06-44ae-9356-31efc9cf7760 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:16:21.632574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1215/2453] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:16:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:18:07.690348 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6048dd-f224-4c96-a93c-b100b993e9ad service placement] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:18:07.699900 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6048dd-f224-4c96-a93c-b100b993e9ad service placement] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:18:07.703677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d8fc0-46b3-4874-8fb4-3eef49536d15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:18:07.704320 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d8fc0-46b3-4874-8fb4-3eef49536d15 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:18:07.704591 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d8fc0-46b3-4874-8fb4-3eef49536d15 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:18:07.704889 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d8fc0-46b3-4874-8fb4-3eef49536d15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:18:07.714634 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d8fc0-46b3-4874-8fb4-3eef49536d15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:18:07.716427 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d8fc0-46b3-4874-8fb4-3eef49536d15 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:18:07.744935 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1239/2454] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:18:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:18:09.485667 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76802e-4df2-4b45-89ea-46a8cae86d93 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:18:09.486217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76802e-4df2-4b45-89ea-46a8cae86d93 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:18:09.486327 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76802e-4df2-4b45-89ea-46a8cae86d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:18:09.515822 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-1f76802e-4df2-4b45-89ea-46a8cae86d93 None None] MFA Rules not processed for user `f89c5ab1020947a9bece03af4c237480`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:18:09.517222 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-1f76802e-4df2-4b45-89ea-46a8cae86d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:18:09.525721 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1216/2455] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:18:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:18:12.749328 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d8fc0-46b3-4874-8fb4-3eef49536d15 service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:18:12.763620 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d8fc0-46b3-4874-8fb4-3eef49536d15 service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:18:12.770655 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac6da6c-b537-4efb-84e0-5af2a16de191 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:18:12.771517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6da6c-b537-4efb-84e0-5af2a16de191 service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:18:12.772102 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6da6c-b537-4efb-84e0-5af2a16de191 service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:18:12.772398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6da6c-b537-4efb-84e0-5af2a16de191 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:18:12.787102 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac6da6c-b537-4efb-84e0-5af2a16de191 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:18:12.789428 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac6da6c-b537-4efb-84e0-5af2a16de191 service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:18:12.801830 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1240/2456] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:18:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:18:43.064933 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76802e-4df2-4b45-89ea-46a8cae86d93 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:18:43.076907 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76802e-4df2-4b45-89ea-46a8cae86d93 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:18:43.081665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19528ca-f53d-443c-a424-f5b3a1fddbf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:18:43.082373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19528ca-f53d-443c-a424-f5b3a1fddbf5 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:18:43.082692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19528ca-f53d-443c-a424-f5b3a1fddbf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:18:43.082959 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19528ca-f53d-443c-a424-f5b3a1fddbf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:18:43.092525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19528ca-f53d-443c-a424-f5b3a1fddbf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:18:43.094119 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19528ca-f53d-443c-a424-f5b3a1fddbf5 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:18:43.104188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1217/2457] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:18:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:19:07.065074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac6da6c-b537-4efb-84e0-5af2a16de191 service nova] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:19:07.077012 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac6da6c-b537-4efb-84e0-5af2a16de191 service nova] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:19:07.084061 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd326533-330c-462e-99c4-740ee9e010d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:19:07.084691 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd326533-330c-462e-99c4-740ee9e010d0 service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:19:07.085013 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd326533-330c-462e-99c4-740ee9e010d0 service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:19:07.085279 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd326533-330c-462e-99c4-740ee9e010d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:19:07.094266 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd326533-330c-462e-99c4-740ee9e010d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:19:07.095854 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd326533-330c-462e-99c4-740ee9e010d0 service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:19:07.124521 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1241/2458] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:19:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:20:08.034909 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19528ca-f53d-443c-a424-f5b3a1fddbf5 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:20:08.065160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19528ca-f53d-443c-a424-f5b3a1fddbf5 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:20:08.069596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b26c9-449d-44c2-b571-617420897296 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:20:08.070120 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b26c9-449d-44c2-b571-617420897296 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:20:08.070371 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b26c9-449d-44c2-b571-617420897296 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:20:08.070640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b26c9-449d-44c2-b571-617420897296 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:20:08.096138 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b26c9-449d-44c2-b571-617420897296 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:20:08.097731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b26c9-449d-44c2-b571-617420897296 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:20:08.108080 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1218/2459] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:20:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:20:08.484651 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd326533-330c-462e-99c4-740ee9e010d0 service nova] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:20:08.504772 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd326533-330c-462e-99c4-740ee9e010d0 service nova] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:20:08.508758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47031b03-6c3c-488d-99c7-8ad7c1d2984c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:20:08.509407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47031b03-6c3c-488d-99c7-8ad7c1d2984c service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:20:08.509688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47031b03-6c3c-488d-99c7-8ad7c1d2984c service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:20:08.509961 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47031b03-6c3c-488d-99c7-8ad7c1d2984c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:20:08.527656 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47031b03-6c3c-488d-99c7-8ad7c1d2984c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:20:08.529159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47031b03-6c3c-488d-99c7-8ad7c1d2984c service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:20:08.541208 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1242/2460] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:20:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:20:09.713856 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b26c9-449d-44c2-b571-617420897296 service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:20:09.717318 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-b25b26c9-449d-44c2-b571-617420897296 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:20:09.735247 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47031b03-6c3c-488d-99c7-8ad7c1d2984c service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:20:09.746710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47031b03-6c3c-488d-99c7-8ad7c1d2984c service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:20:09.752134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd225c15-8f3f-4d59-9d9c-37571c8a12b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:20:09.753014 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd225c15-8f3f-4d59-9d9c-37571c8a12b5 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:20:09.753364 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd225c15-8f3f-4d59-9d9c-37571c8a12b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:20:09.753710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd225c15-8f3f-4d59-9d9c-37571c8a12b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:20:09.764639 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b26c9-449d-44c2-b571-617420897296 service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:20:09.767187 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd225c15-8f3f-4d59-9d9c-37571c8a12b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:20:09.768772 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd225c15-8f3f-4d59-9d9c-37571c8a12b5 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:20:09.769167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a41119-5d94-4ec2-bcee-c4210a6a47b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:20:09.769837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a41119-5d94-4ec2-bcee-c4210a6a47b4 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:20:09.770113 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a41119-5d94-4ec2-bcee-c4210a6a47b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:20:09.770385 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a41119-5d94-4ec2-bcee-c4210a6a47b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:20:09.779592 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a41119-5d94-4ec2-bcee-c4210a6a47b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:20:09.781168 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a41119-5d94-4ec2-bcee-c4210a6a47b4 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:20:09.785423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1243/2461] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:20:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 10:20:09.820858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1219/2462] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:20:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:20:09.982554 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd225c15-8f3f-4d59-9d9c-37571c8a12b5 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:20:09.992273 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd225c15-8f3f-4d59-9d9c-37571c8a12b5 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:20:09.996105 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13c1e6-ff20-42a6-9093-cc323b789262 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:20:09.996986 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13c1e6-ff20-42a6-9093-cc323b789262 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:20:09.997451 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13c1e6-ff20-42a6-9093-cc323b789262 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:20:09.997858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13c1e6-ff20-42a6-9093-cc323b789262 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:20:10.007826 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13c1e6-ff20-42a6-9093-cc323b789262 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:20:10.009327 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13c1e6-ff20-42a6-9093-cc323b789262 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:20:10.019683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1244/2463] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:20:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:20:49.396611 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a41119-5d94-4ec2-bcee-c4210a6a47b4 service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:20:49.472798 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a41119-5d94-4ec2-bcee-c4210a6a47b4 service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:20:49.478205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27620d4-3816-40a9-9d67-5505e53bd9f7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9e458c2c5f4ed1b3eddd84c2ec1703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:20:49.478828 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27620d4-3816-40a9-9d67-5505e53bd9f7 service placement] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:20:49.479125 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27620d4-3816-40a9-9d67-5505e53bd9f7 service placement] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:20:49.479390 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27620d4-3816-40a9-9d67-5505e53bd9f7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:20:49.481376 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a27620d4-3816-40a9-9d67-5505e53bd9f7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:20:49.506388 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27620d4-3816-40a9-9d67-5505e53bd9f7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:20:49.507981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27620d4-3816-40a9-9d67-5505e53bd9f7 service placement] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:20:49.518682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1220/2464] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:20:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:23:49.077484 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13c1e6-ff20-42a6-9093-cc323b789262 service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:23:49.079676 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-8e13c1e6-ff20-42a6-9093-cc323b789262 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:23:49.161499 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13c1e6-ff20-42a6-9093-cc323b789262 service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:23:49.166301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a54299b-543a-46b0-b720-384f577ff936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:23:49.166950 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54299b-543a-46b0-b720-384f577ff936 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:23:49.167224 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54299b-543a-46b0-b720-384f577ff936 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:23:49.167489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54299b-543a-46b0-b720-384f577ff936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:23:49.169669 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-1a54299b-543a-46b0-b720-384f577ff936 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:23:49.180406 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a54299b-543a-46b0-b720-384f577ff936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:23:49.181893 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a54299b-543a-46b0-b720-384f577ff936 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:23:49.190076 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1245/2465] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:23:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:24:04.546695 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27620d4-3816-40a9-9d67-5505e53bd9f7 service placement] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:04.558092 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27620d4-3816-40a9-9d67-5505e53bd9f7 service placement] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:04.567439 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811d846-5301-4eae-88d5-091918ecf1d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:04.568332 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811d846-5301-4eae-88d5-091918ecf1d0 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:04.568830 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811d846-5301-4eae-88d5-091918ecf1d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:04.569271 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811d846-5301-4eae-88d5-091918ecf1d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:04.571528 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-1811d846-5301-4eae-88d5-091918ecf1d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:04.656238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1811d846-5301-4eae-88d5-091918ecf1d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:04.658202 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1811d846-5301-4eae-88d5-091918ecf1d0 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:04.669434 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1221/2466] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:24:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:24:07.790956 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a54299b-543a-46b0-b720-384f577ff936 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:07.792533 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-1a54299b-543a-46b0-b720-384f577ff936 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:07.879441 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a54299b-543a-46b0-b720-384f577ff936 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:07.885934 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe0f9c-3315-464d-ac79-3752b73a84db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:07.886657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe0f9c-3315-464d-ac79-3752b73a84db service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:07.887005 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe0f9c-3315-464d-ac79-3752b73a84db service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:07.887271 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe0f9c-3315-464d-ac79-3752b73a84db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:07.889176 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-5dbe0f9c-3315-464d-ac79-3752b73a84db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:07.901972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe0f9c-3315-464d-ac79-3752b73a84db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:07.903597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe0f9c-3315-464d-ac79-3752b73a84db service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:07.914829 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1246/2467] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:24:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:24:08.160494 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811d846-5301-4eae-88d5-091918ecf1d0 service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:08.162929 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-1811d846-5301-4eae-88d5-091918ecf1d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:08.246150 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811d846-5301-4eae-88d5-091918ecf1d0 service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:08.249818 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c26eb-c72f-4404-93cc-a39506df1892 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:08.250466 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c26eb-c72f-4404-93cc-a39506df1892 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:08.250743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c26eb-c72f-4404-93cc-a39506df1892 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:08.251018 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c26eb-c72f-4404-93cc-a39506df1892 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:08.252976 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f75c26eb-c72f-4404-93cc-a39506df1892 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:08.335345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75c26eb-c72f-4404-93cc-a39506df1892 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:08.337267 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75c26eb-c72f-4404-93cc-a39506df1892 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:08.351880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1222/2468] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:24:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:24:09.556227 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe0f9c-3315-464d-ac79-3752b73a84db service nova] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:09.567273 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe0f9c-3315-464d-ac79-3752b73a84db service nova] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:09.571255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c33b7-a63c-435e-8ed8-2a8c4cbdead1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:09.572017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c33b7-a63c-435e-8ed8-2a8c4cbdead1 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:09.572494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c33b7-a63c-435e-8ed8-2a8c4cbdead1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:09.572812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c33b7-a63c-435e-8ed8-2a8c4cbdead1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:09.574634 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-892c33b7-a63c-435e-8ed8-2a8c4cbdead1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:09.620239 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c33b7-a63c-435e-8ed8-2a8c4cbdead1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:09.622001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c33b7-a63c-435e-8ed8-2a8c4cbdead1 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:09.642368 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1247/2469] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:24:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:24:09.909279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc4ffd-1290-4f3b-a333-e3e3d0853a32 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:09.909776 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc4ffd-1290-4f3b-a333-e3e3d0853a32 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:09.910095 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc4ffd-1290-4f3b-a333-e3e3d0853a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:09.972387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-84dc4ffd-1290-4f3b-a333-e3e3d0853a32 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:24:09.973822 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-84dc4ffd-1290-4f3b-a333-e3e3d0853a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:09.982293 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1223/2470] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 10:24:09.990155 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459c7e2-dc92-433a-b19e-ae7d74782bde None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:09.990597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459c7e2-dc92-433a-b19e-ae7d74782bde None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:09.991046 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459c7e2-dc92-433a-b19e-ae7d74782bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:10.012588 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-5459c7e2-dc92-433a-b19e-ae7d74782bde None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:24:10.013848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-5459c7e2-dc92-433a-b19e-ae7d74782bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:10.023081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1248/2471] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:24:10.028535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dc4ffd-1290-4f3b-a333-e3e3d0853a32 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:10.042490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dc4ffd-1290-4f3b-a333-e3e3d0853a32 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:10.049945 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8503a-f267-456e-bb5c-eeead2ce76bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:10.050713 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8503a-f267-456e-bb5c-eeead2ce76bc admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:10.050989 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8503a-f267-456e-bb5c-eeead2ce76bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:10.051348 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8503a-f267-456e-bb5c-eeead2ce76bc admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:10.052102 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8503a-f267-456e-bb5c-eeead2ce76bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 10:24:10.053082 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8503a-f267-456e-bb5c-eeead2ce76bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:10.054754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8503a-f267-456e-bb5c-eeead2ce76bc admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:10.067717 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1224/2472] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 10:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 10:24:10.072517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459c7e2-dc92-433a-b19e-ae7d74782bde None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:10.082220 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459c7e2-dc92-433a-b19e-ae7d74782bde None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:10.088351 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a57768-8a18-4d9b-9595-78fd2aec728e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:10.089990 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a57768-8a18-4d9b-9595-78fd2aec728e admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:10.090274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a57768-8a18-4d9b-9595-78fd2aec728e admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:10.090549 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a57768-8a18-4d9b-9595-78fd2aec728e admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:10.091277 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a57768-8a18-4d9b-9595-78fd2aec728e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:10.093230 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a57768-8a18-4d9b-9595-78fd2aec728e admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:10.132584 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1249/2473] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:24:10 2022] POST /identity/v3/projects => generated 389 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:24:10.136963 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8503a-f267-456e-bb5c-eeead2ce76bc admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:10.147061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8503a-f267-456e-bb5c-eeead2ce76bc admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:10.208718 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e922f8-8049-43b6-aa54-b58ef62d35eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:10.209415 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e922f8-8049-43b6-aa54-b58ef62d35eb admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:10.209835 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e922f8-8049-43b6-aa54-b58ef62d35eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:10.210297 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e922f8-8049-43b6-aa54-b58ef62d35eb admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:10.211071 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e922f8-8049-43b6-aa54-b58ef62d35eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:10.213524 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e922f8-8049-43b6-aa54-b58ef62d35eb admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:10.237945 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1225/2474] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:24:10 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 10:24:10.244527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a57768-8a18-4d9b-9595-78fd2aec728e admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:10.260135 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a57768-8a18-4d9b-9595-78fd2aec728e admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:10.265027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347601e4-c4cd-446b-a31d-63803ef4cf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:10.265699 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347601e4-c4cd-446b-a31d-63803ef4cf53 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:10.265989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347601e4-c4cd-446b-a31d-63803ef4cf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:10.266254 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347601e4-c4cd-446b-a31d-63803ef4cf53 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:10.267018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347601e4-c4cd-446b-a31d-63803ef4cf53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:10.268700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347601e4-c4cd-446b-a31d-63803ef4cf53 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:10.280230 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1250/2475] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:24:10.284655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e922f8-8049-43b6-aa54-b58ef62d35eb admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:10.295697 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e922f8-8049-43b6-aa54-b58ef62d35eb admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:10.301274 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d887b-07a5-4040-a32e-dc0b1750e811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:10.301943 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d887b-07a5-4040-a32e-dc0b1750e811 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:10.302225 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d887b-07a5-4040-a32e-dc0b1750e811 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:10.302492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d887b-07a5-4040-a32e-dc0b1750e811 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:10.303307 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d887b-07a5-4040-a32e-dc0b1750e811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:10.304963 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d887b-07a5-4040-a32e-dc0b1750e811 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:10.319423 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1226/2476] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:24:10.321215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347601e4-c4cd-446b-a31d-63803ef4cf53 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:10.331976 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347601e4-c4cd-446b-a31d-63803ef4cf53 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:10.337451 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af82a4-7466-47e5-82f3-c33d0d8a979c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:10.338340 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af82a4-7466-47e5-82f3-c33d0d8a979c admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:10.338616 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af82a4-7466-47e5-82f3-c33d0d8a979c admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:10.338882 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af82a4-7466-47e5-82f3-c33d0d8a979c admin admin] PATH_INFO: `/v3/projects/38b476edcb0344c3801c7ea990930f7b/users/662f4299543f4407b333cdccdaeb9090/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:10.355482 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af82a4-7466-47e5-82f3-c33d0d8a979c admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b476edcb0344c3801c7ea990930f7b, user_id=662f4299543f4407b333cdccdaeb9090, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:10.358064 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af82a4-7466-47e5-82f3-c33d0d8a979c admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:10.374956 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1251/2477] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:24:10 2022] PUT /identity/v3/projects/38b476edcb0344c3801c7ea990930f7b/users/662f4299543f4407b333cdccdaeb9090/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:24:10.389386 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6ff42-f32f-4edf-b764-3fda151a879a None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:10.389606 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6ff42-f32f-4edf-b764-3fda151a879a None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:10.389882 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6ff42-f32f-4edf-b764-3fda151a879a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:10.426584 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-f4a6ff42-f32f-4edf-b764-3fda151a879a None None] MFA Rules not processed for user `662f4299543f4407b333cdccdaeb9090`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:24:10.427895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f4a6ff42-f32f-4edf-b764-3fda151a879a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:10.510499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1227/2478] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:24:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:24:10.521421 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af82a4-7466-47e5-82f3-c33d0d8a979c admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:10.578753 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af82a4-7466-47e5-82f3-c33d0d8a979c admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:10.582750 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f83973-5f1b-431e-b028-dabf3eaeb9f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:10.583423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f83973-5f1b-431e-b028-dabf3eaeb9f7 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:10.583730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f83973-5f1b-431e-b028-dabf3eaeb9f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:10.584004 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f83973-5f1b-431e-b028-dabf3eaeb9f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:10.593283 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f83973-5f1b-431e-b028-dabf3eaeb9f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:10.594752 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f83973-5f1b-431e-b028-dabf3eaeb9f7 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:10.603915 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1252/2479] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:24:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:24:32.508241 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6ff42-f32f-4edf-b764-3fda151a879a None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:32.510180 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f4a6ff42-f32f-4edf-b764-3fda151a879a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:32.595736 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6ff42-f32f-4edf-b764-3fda151a879a None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:32.601549 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3bc25-4837-43fc-9320-d978b76af6f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:32.602193 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3bc25-4837-43fc-9320-d978b76af6f8 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:32.602474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3bc25-4837-43fc-9320-d978b76af6f8 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:32.602728 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3bc25-4837-43fc-9320-d978b76af6f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:32.612014 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3bc25-4837-43fc-9320-d978b76af6f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:32.613639 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3bc25-4837-43fc-9320-d978b76af6f8 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:32.624759 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1228/2480] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:24:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:24:32.956454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f83973-5f1b-431e-b028-dabf3eaeb9f7 service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:33.022407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f83973-5f1b-431e-b028-dabf3eaeb9f7 service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:33.027246 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb7daa-3292-4560-b2cd-8c803893b177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:33.028125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb7daa-3292-4560-b2cd-8c803893b177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:33.028397 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb7daa-3292-4560-b2cd-8c803893b177 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:33.028682 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb7daa-3292-4560-b2cd-8c803893b177 admin admin] PATH_INFO: `/v3/users/662f4299543f4407b333cdccdaeb9090` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:33.029903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eb7daa-3292-4560-b2cd-8c803893b177 admin admin] RBAC: Authorizing `identity:delete_user(user_id=662f4299543f4407b333cdccdaeb9090)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:33.031851 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eb7daa-3292-4560-b2cd-8c803893b177 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:33.085016 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-b2eb7daa-3292-4560-b2cd-8c803893b177 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '662f4299543f4407b333cdccdaeb9090'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:33.094180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-b2eb7daa-3292-4560-b2cd-8c803893b177 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '662f4299543f4407b333cdccdaeb9090'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:33.096217 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-b2eb7daa-3292-4560-b2cd-8c803893b177 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '662f4299543f4407b333cdccdaeb9090'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:33.106042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-b2eb7daa-3292-4560-b2cd-8c803893b177 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '662f4299543f4407b333cdccdaeb9090'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:33.106937 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-b2eb7daa-3292-4560-b2cd-8c803893b177 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '662f4299543f4407b333cdccdaeb9090'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:33.128867 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1253/2481] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:24:32 2022] DELETE /identity/v3/users/662f4299543f4407b333cdccdaeb9090 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:24:33.134142 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8478a-6f83-43b3-93a9-205199dba216 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:33.134471 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8478a-6f83-43b3-93a9-205199dba216 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:33.134689 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8478a-6f83-43b3-93a9-205199dba216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:33.154181 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-ada8478a-6f83-43b3-93a9-205199dba216 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:24:33.155649 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-ada8478a-6f83-43b3-93a9-205199dba216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:33.221755 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1229/2482] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:24:33.231644 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb7daa-3292-4560-b2cd-8c803893b177 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:33.233536 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b2eb7daa-3292-4560-b2cd-8c803893b177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:33.291278 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb7daa-3292-4560-b2cd-8c803893b177 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:33.295469 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421e641-04b7-447f-a8f2-0fcaa8fc4519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:33.296152 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421e641-04b7-447f-a8f2-0fcaa8fc4519 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:33.296431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421e641-04b7-447f-a8f2-0fcaa8fc4519 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:33.296693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421e641-04b7-447f-a8f2-0fcaa8fc4519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:33.306682 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9421e641-04b7-447f-a8f2-0fcaa8fc4519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:33.308107 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9421e641-04b7-447f-a8f2-0fcaa8fc4519 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:33.318920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1254/2483] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:24:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:24:33.684734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8478a-6f83-43b3-93a9-205199dba216 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:33.686632 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-ada8478a-6f83-43b3-93a9-205199dba216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:33.704779 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8478a-6f83-43b3-93a9-205199dba216 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:33.710683 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f695145-1caa-469e-b423-496b00758ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:33.712129 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f695145-1caa-469e-b423-496b00758ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:33.712623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f695145-1caa-469e-b423-496b00758ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:33.713106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f695145-1caa-469e-b423-496b00758ecc admin admin] PATH_INFO: `/v3/projects/38b476edcb0344c3801c7ea990930f7b` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:33.715463 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f695145-1caa-469e-b423-496b00758ecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=38b476edcb0344c3801c7ea990930f7b)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:33.719322 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f695145-1caa-469e-b423-496b00758ecc admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:33.778552 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4f695145-1caa-469e-b423-496b00758ecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38b476edcb0344c3801c7ea990930f7b', 'request_id': 'req-4f695145-1caa-469e-b423-496b00758ecc'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:33.787860 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4f695145-1caa-469e-b423-496b00758ecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38b476edcb0344c3801c7ea990930f7b', 'request_id': 'req-4f695145-1caa-469e-b423-496b00758ecc'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:33.793949 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4f695145-1caa-469e-b423-496b00758ecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38b476edcb0344c3801c7ea990930f7b', 'request_id': 'req-4f695145-1caa-469e-b423-496b00758ecc'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:33.808537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4f695145-1caa-469e-b423-496b00758ecc admin admin] The token cache is being invalidate because project 38b476edcb0344c3801c7ea990930f7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:24:33.808858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4f695145-1caa-469e-b423-496b00758ecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:33.809524 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4f695145-1caa-469e-b423-496b00758ecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:33.811341 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1230/2484] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:24:33 2022] DELETE /identity/v3/projects/38b476edcb0344c3801c7ea990930f7b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:24:35.471384 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f95b02-04b9-4aaa-a704-1578211a8f78 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:35.471921 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f95b02-04b9-4aaa-a704-1578211a8f78 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:35.471921 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f95b02-04b9-4aaa-a704-1578211a8f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:35.490759 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-05f95b02-04b9-4aaa-a704-1578211a8f78 None None] MFA Rules not processed for user `af62b2d3f73f482b8db4173af33752c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:24:35.491973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-05f95b02-04b9-4aaa-a704-1578211a8f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:35.548878 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1255/2485] 10.222.0.26 () {58 vars in 1009 bytes} [Mon May 9 10:24:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:24:35.601240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f695145-1caa-469e-b423-496b00758ecc admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:35.602700 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-4f695145-1caa-469e-b423-496b00758ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:35.673068 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f695145-1caa-469e-b423-496b00758ecc admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:35.679044 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7207a4f-e798-4724-a84b-d367361e8dac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380e04358f2349aa912d6a97518877cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:35.679709 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7207a4f-e798-4724-a84b-d367361e8dac service swift] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:35.679991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7207a4f-e798-4724-a84b-d367361e8dac service swift] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:35.681599 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7207a4f-e798-4724-a84b-d367361e8dac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:35.690211 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7207a4f-e798-4724-a84b-d367361e8dac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:35.692643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7207a4f-e798-4724-a84b-d367361e8dac service swift] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:35.701685 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1231/2486] 10.222.0.26 () {60 vars in 1384 bytes} [Mon May 9 10:24:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 10:24:42.661596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f95b02-04b9-4aaa-a704-1578211a8f78 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:42.663307 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-05f95b02-04b9-4aaa-a704-1578211a8f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:42.729804 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f95b02-04b9-4aaa-a704-1578211a8f78 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:42.736692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc143f0e-b858-4515-975a-e5b1d20c5d71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:42.737778 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc143f0e-b858-4515-975a-e5b1d20c5d71 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:42.738217 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc143f0e-b858-4515-975a-e5b1d20c5d71 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:42.738667 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc143f0e-b858-4515-975a-e5b1d20c5d71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:42.741576 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-dc143f0e-b858-4515-975a-e5b1d20c5d71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:42.819822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc143f0e-b858-4515-975a-e5b1d20c5d71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:42.821796 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc143f0e-b858-4515-975a-e5b1d20c5d71 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:42.833613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1256/2487] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:24:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 10:24:46.729714 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7207a4f-e798-4724-a84b-d367361e8dac service swift] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:46.731175 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f7207a4f-e798-4724-a84b-d367361e8dac service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:46.746049 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7207a4f-e798-4724-a84b-d367361e8dac service swift] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:46.750462 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5187a-fbc7-492e-8d1b-dcbc06d0cdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:46.751333 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5187a-fbc7-492e-8d1b-dcbc06d0cdc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:46.751610 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5187a-fbc7-492e-8d1b-dcbc06d0cdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:46.751881 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5187a-fbc7-492e-8d1b-dcbc06d0cdc3 admin admin] PATH_INFO: `/v3/users/f89c5ab1020947a9bece03af4c237480` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:46.753117 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5187a-fbc7-492e-8d1b-dcbc06d0cdc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f89c5ab1020947a9bece03af4c237480)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:46.755124 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5187a-fbc7-492e-8d1b-dcbc06d0cdc3 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:46.810747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4ba5187a-fbc7-492e-8d1b-dcbc06d0cdc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f89c5ab1020947a9bece03af4c237480'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:46.818176 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4ba5187a-fbc7-492e-8d1b-dcbc06d0cdc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f89c5ab1020947a9bece03af4c237480'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:46.827115 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4ba5187a-fbc7-492e-8d1b-dcbc06d0cdc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f89c5ab1020947a9bece03af4c237480'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:46.827754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4ba5187a-fbc7-492e-8d1b-dcbc06d0cdc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f89c5ab1020947a9bece03af4c237480'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:46.828587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4ba5187a-fbc7-492e-8d1b-dcbc06d0cdc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f89c5ab1020947a9bece03af4c237480'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:46.850392 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1232/2488] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:24:46 2022] DELETE /identity/v3/users/f89c5ab1020947a9bece03af4c237480 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:24:47.027295 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc143f0e-b858-4515-975a-e5b1d20c5d71 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:47.037858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc143f0e-b858-4515-975a-e5b1d20c5d71 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:47.099331 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef69b6e0-04d4-4679-8e99-a114f5cae8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:47.100229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69b6e0-04d4-4679-8e99-a114f5cae8b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:47.100509 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69b6e0-04d4-4679-8e99-a114f5cae8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:47.100820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69b6e0-04d4-4679-8e99-a114f5cae8b5 admin admin] PATH_INFO: `/v3/projects/c90b9a64e6a8464b9cde8bb967c4872d` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:47.102024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef69b6e0-04d4-4679-8e99-a114f5cae8b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c90b9a64e6a8464b9cde8bb967c4872d)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:47.104089 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef69b6e0-04d4-4679-8e99-a114f5cae8b5 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:47.156280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-ef69b6e0-04d4-4679-8e99-a114f5cae8b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c90b9a64e6a8464b9cde8bb967c4872d', 'request_id': 'req-ef69b6e0-04d4-4679-8e99-a114f5cae8b5'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:47.163857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-ef69b6e0-04d4-4679-8e99-a114f5cae8b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c90b9a64e6a8464b9cde8bb967c4872d', 'request_id': 'req-ef69b6e0-04d4-4679-8e99-a114f5cae8b5'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:47.170013 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-ef69b6e0-04d4-4679-8e99-a114f5cae8b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c90b9a64e6a8464b9cde8bb967c4872d', 'request_id': 'req-ef69b6e0-04d4-4679-8e99-a114f5cae8b5'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:47.184976 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-ef69b6e0-04d4-4679-8e99-a114f5cae8b5 admin admin] The token cache is being invalidate because project c90b9a64e6a8464b9cde8bb967c4872d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:24:47.185260 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-ef69b6e0-04d4-4679-8e99-a114f5cae8b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:47.185885 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-ef69b6e0-04d4-4679-8e99-a114f5cae8b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:47.187472 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1257/2489] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:24:47 2022] DELETE /identity/v3/projects/c90b9a64e6a8464b9cde8bb967c4872d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:24:47.192647 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8002ce0-bbf1-477b-b2a6-70017a3a8752 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:47.192966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8002ce0-bbf1-477b-b2a6-70017a3a8752 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:47.193238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8002ce0-bbf1-477b-b2a6-70017a3a8752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:47.211788 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-c8002ce0-bbf1-477b-b2a6-70017a3a8752 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:24:47.213053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-c8002ce0-bbf1-477b-b2a6-70017a3a8752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:47.289295 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1233/2490] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:24:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:24:47.297494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4ab36-4273-42ff-a158-e0c078dc7c3e None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:47.297629 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4ab36-4273-42ff-a158-e0c078dc7c3e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:47.297906 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4ab36-4273-42ff-a158-e0c078dc7c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:47.317148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-59b4ab36-4273-42ff-a158-e0c078dc7c3e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:24:47.318355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-59b4ab36-4273-42ff-a158-e0c078dc7c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:47.326625 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1258/2491] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:24:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:24:47.330959 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8002ce0-bbf1-477b-b2a6-70017a3a8752 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:47.341390 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8002ce0-bbf1-477b-b2a6-70017a3a8752 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:47.345988 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a9ca1-835a-4f85-8ee6-f60c2acb8ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:47.346642 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a9ca1-835a-4f85-8ee6-f60c2acb8ab1 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:47.346915 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a9ca1-835a-4f85-8ee6-f60c2acb8ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:47.349723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a9ca1-835a-4f85-8ee6-f60c2acb8ab1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:47.349723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a9ca1-835a-4f85-8ee6-f60c2acb8ab1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 10:24:47.349723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a9ca1-835a-4f85-8ee6-f60c2acb8ab1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:47.350173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a9ca1-835a-4f85-8ee6-f60c2acb8ab1 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:47.362073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1234/2492] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 10:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 10:24:47.366185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4ab36-4273-42ff-a158-e0c078dc7c3e None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:47.375341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4ab36-4273-42ff-a158-e0c078dc7c3e None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:47.380459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c1117-c512-4cac-b532-9e90e5a2f1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:47.381136 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c1117-c512-4cac-b532-9e90e5a2f1d7 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:47.381409 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c1117-c512-4cac-b532-9e90e5a2f1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:47.381677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c1117-c512-4cac-b532-9e90e5a2f1d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:47.382357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c1117-c512-4cac-b532-9e90e5a2f1d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:47.384266 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c1117-c512-4cac-b532-9e90e5a2f1d7 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:47.417779 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1259/2493] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:24:47 2022] POST /identity/v3/projects => generated 405 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 10:24:47.421664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a9ca1-835a-4f85-8ee6-f60c2acb8ab1 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:47.432374 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a9ca1-835a-4f85-8ee6-f60c2acb8ab1 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:47.493601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fa67b-dbb5-4493-b2d2-60532ee5a5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:47.494270 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fa67b-dbb5-4493-b2d2-60532ee5a5d5 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:47.494542 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fa67b-dbb5-4493-b2d2-60532ee5a5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:47.494811 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fa67b-dbb5-4493-b2d2-60532ee5a5d5 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:47.495492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4fa67b-dbb5-4493-b2d2-60532ee5a5d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:47.497435 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4fa67b-dbb5-4493-b2d2-60532ee5a5d5 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:47.518562 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1235/2494] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:24:47 2022] POST /identity/v3/users => generated 303 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 10:24:47.522792 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c1117-c512-4cac-b532-9e90e5a2f1d7 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:47.533949 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c1117-c512-4cac-b532-9e90e5a2f1d7 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:47.539286 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e28c2-968d-4482-a30a-242765e4b084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:47.539949 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e28c2-968d-4482-a30a-242765e4b084 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:47.540225 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e28c2-968d-4482-a30a-242765e4b084 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:47.540495 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e28c2-968d-4482-a30a-242765e4b084 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:47.541326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e28c2-968d-4482-a30a-242765e4b084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:47.542929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e28c2-968d-4482-a30a-242765e4b084 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:47.555244 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1260/2495] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:24:47.559542 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fa67b-dbb5-4493-b2d2-60532ee5a5d5 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:47.570110 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fa67b-dbb5-4493-b2d2-60532ee5a5d5 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:47.575918 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74baf5d3-7645-4d13-b0c8-9ca11926c6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:47.576609 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74baf5d3-7645-4d13-b0c8-9ca11926c6c3 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:47.576913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74baf5d3-7645-4d13-b0c8-9ca11926c6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:47.577180 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74baf5d3-7645-4d13-b0c8-9ca11926c6c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:47.577903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74baf5d3-7645-4d13-b0c8-9ca11926c6c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:47.579517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74baf5d3-7645-4d13-b0c8-9ca11926c6c3 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:47.590637 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1236/2496] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:24:47.595542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e28c2-968d-4482-a30a-242765e4b084 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:47.611136 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e28c2-968d-4482-a30a-242765e4b084 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:47.618001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a0de1a-24c0-457a-bbfc-630849aa27af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:47.618468 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0de1a-24c0-457a-bbfc-630849aa27af admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:47.618720 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0de1a-24c0-457a-bbfc-630849aa27af admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:47.618991 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0de1a-24c0-457a-bbfc-630849aa27af admin admin] PATH_INFO: `/v3/projects/a7c2e9a8c1204ee5a62a80c6c86ffd78/users/4f8e978e56a441439cf374d43a02970d/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:47.634944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a0de1a-24c0-457a-bbfc-630849aa27af admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7c2e9a8c1204ee5a62a80c6c86ffd78, user_id=4f8e978e56a441439cf374d43a02970d, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:47.637729 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a0de1a-24c0-457a-bbfc-630849aa27af admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:47.651632 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1261/2497] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:24:47 2022] PUT /identity/v3/projects/a7c2e9a8c1204ee5a62a80c6c86ffd78/users/4f8e978e56a441439cf374d43a02970d/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:24:47.657031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d135c7-ae82-49b0-a6d5-6586243e16ed None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:47.657406 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d135c7-ae82-49b0-a6d5-6586243e16ed None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:47.657802 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d135c7-ae82-49b0-a6d5-6586243e16ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:47.678039 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-90d135c7-ae82-49b0-a6d5-6586243e16ed None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:24:47.679226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-90d135c7-ae82-49b0-a6d5-6586243e16ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:47.744577 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1237/2498] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:24:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:24:47.756302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a0de1a-24c0-457a-bbfc-630849aa27af admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:47.758240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-98a0de1a-24c0-457a-bbfc-630849aa27af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:47.824269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a0de1a-24c0-457a-bbfc-630849aa27af admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:47.828611 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52b11f-fbee-40b8-9027-55d2041f3a43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:47.829379 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52b11f-fbee-40b8-9027-55d2041f3a43 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:47.829640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52b11f-fbee-40b8-9027-55d2041f3a43 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:47.829913 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52b11f-fbee-40b8-9027-55d2041f3a43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:47.839697 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52b11f-fbee-40b8-9027-55d2041f3a43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:47.841187 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52b11f-fbee-40b8-9027-55d2041f3a43 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:47.850957 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1262/2499] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:24:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:24:48.506710 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66364a5e-543e-4bd1-ba7b-5febea0b4f1a None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:48.506994 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66364a5e-543e-4bd1-ba7b-5febea0b4f1a None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:48.507261 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66364a5e-543e-4bd1-ba7b-5febea0b4f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:48.541404 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-66364a5e-543e-4bd1-ba7b-5febea0b4f1a None None] MFA Rules not processed for user `4f8e978e56a441439cf374d43a02970d`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:24:48.541404 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-66364a5e-543e-4bd1-ba7b-5febea0b4f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:48.613792 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1238/2500] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:24:48 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:24:48.614832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52b11f-fbee-40b8-9027-55d2041f3a43 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:48.624413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52b11f-fbee-40b8-9027-55d2041f3a43 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:48.629541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765c738-51bf-4daa-82f6-9fea258d879d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:48.630202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765c738-51bf-4daa-82f6-9fea258d879d admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:48.630472 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765c738-51bf-4daa-82f6-9fea258d879d admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:48.630922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765c738-51bf-4daa-82f6-9fea258d879d admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:48.631629 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765c738-51bf-4daa-82f6-9fea258d879d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:48.633567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765c738-51bf-4daa-82f6-9fea258d879d admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:48.661126 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1263/2501] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:24:48 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:24:48.665357 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66364a5e-543e-4bd1-ba7b-5febea0b4f1a None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:48.676689 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66364a5e-543e-4bd1-ba7b-5febea0b4f1a None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:48.734112 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79210b-ea90-48b4-970f-8d81e2136f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:48.734685 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79210b-ea90-48b4-970f-8d81e2136f72 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:48.734966 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79210b-ea90-48b4-970f-8d81e2136f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:48.735234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79210b-ea90-48b4-970f-8d81e2136f72 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:48.735956 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79210b-ea90-48b4-970f-8d81e2136f72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:48.738057 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79210b-ea90-48b4-970f-8d81e2136f72 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:48.759414 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1239/2502] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:24:48 2022] POST /identity/v3/users => generated 308 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:24:48.763647 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765c738-51bf-4daa-82f6-9fea258d879d admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:48.773997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765c738-51bf-4daa-82f6-9fea258d879d admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:48.778826 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9eefd-88c0-4f70-8b2c-6bdbf6635519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:48.779564 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9eefd-88c0-4f70-8b2c-6bdbf6635519 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:48.779903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9eefd-88c0-4f70-8b2c-6bdbf6635519 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:48.780234 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9eefd-88c0-4f70-8b2c-6bdbf6635519 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:48.781088 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9eefd-88c0-4f70-8b2c-6bdbf6635519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:48.782873 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9eefd-88c0-4f70-8b2c-6bdbf6635519 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:48.795496 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1264/2503] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 10:24:48.801980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79210b-ea90-48b4-970f-8d81e2136f72 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:48.813314 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79210b-ea90-48b4-970f-8d81e2136f72 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:48.818053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc9aa3b-ccf0-4b41-93d0-1dd90aed7f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:48.818986 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9aa3b-ccf0-4b41-93d0-1dd90aed7f55 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:48.819254 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9aa3b-ccf0-4b41-93d0-1dd90aed7f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:48.819519 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9aa3b-ccf0-4b41-93d0-1dd90aed7f55 admin admin] PATH_INFO: `/v3/projects/e18b57fe7a0f47598dbfcf34c43acc24/users/5901fc063f874a6c91cbf9e3b6183944/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:48.835158 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc9aa3b-ccf0-4b41-93d0-1dd90aed7f55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e18b57fe7a0f47598dbfcf34c43acc24, user_id=5901fc063f874a6c91cbf9e3b6183944, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:48.837846 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc9aa3b-ccf0-4b41-93d0-1dd90aed7f55 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:48.853388 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1240/2504] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:24:48 2022] PUT /identity/v3/projects/e18b57fe7a0f47598dbfcf34c43acc24/users/5901fc063f874a6c91cbf9e3b6183944/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:24:48.857510 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9eefd-88c0-4f70-8b2c-6bdbf6635519 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:48.867268 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9eefd-88c0-4f70-8b2c-6bdbf6635519 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:48.929786 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68300c-b14f-4623-a062-adb80e7c2a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:48.930934 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68300c-b14f-4623-a062-adb80e7c2a35 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:48.931383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68300c-b14f-4623-a062-adb80e7c2a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:48.931829 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68300c-b14f-4623-a062-adb80e7c2a35 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:48.933095 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68300c-b14f-4623-a062-adb80e7c2a35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:48.935847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68300c-b14f-4623-a062-adb80e7c2a35 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:48.947410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1265/2505] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:24:48.951690 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc9aa3b-ccf0-4b41-93d0-1dd90aed7f55 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:48.964347 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc9aa3b-ccf0-4b41-93d0-1dd90aed7f55 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:48.971148 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261737fc-e4e5-4817-a9dd-6e50a57fa404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:48.972114 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261737fc-e4e5-4817-a9dd-6e50a57fa404 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:48.972432 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261737fc-e4e5-4817-a9dd-6e50a57fa404 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:48.972714 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261737fc-e4e5-4817-a9dd-6e50a57fa404 admin admin] PATH_INFO: `/v3/projects/e18b57fe7a0f47598dbfcf34c43acc24/users/5901fc063f874a6c91cbf9e3b6183944/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:48.974846 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261737fc-e4e5-4817-a9dd-6e50a57fa404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e18b57fe7a0f47598dbfcf34c43acc24, user_id=5901fc063f874a6c91cbf9e3b6183944, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:48.977612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261737fc-e4e5-4817-a9dd-6e50a57fa404 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:48.989309 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1241/2506] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:24:48 2022] PUT /identity/v3/projects/e18b57fe7a0f47598dbfcf34c43acc24/users/5901fc063f874a6c91cbf9e3b6183944/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:24:49.504091 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68300c-b14f-4623-a062-adb80e7c2a35 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:49.506119 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-df68300c-b14f-4623-a062-adb80e7c2a35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:49.570303 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68300c-b14f-4623-a062-adb80e7c2a35 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:49.632238 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba7846-38ca-4d49-8801-ec351f5f1d93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:49.633056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba7846-38ca-4d49-8801-ec351f5f1d93 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:49.633573 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba7846-38ca-4d49-8801-ec351f5f1d93 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:49.633912 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba7846-38ca-4d49-8801-ec351f5f1d93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:49.638054 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-55ba7846-38ca-4d49-8801-ec351f5f1d93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:49.646858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97511bc5-cd14-457e-9f47-84977e8d7530 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:49.647279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97511bc5-cd14-457e-9f47-84977e8d7530 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:49.647652 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97511bc5-cd14-457e-9f47-84977e8d7530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:49.650228 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba7846-38ca-4d49-8801-ec351f5f1d93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:49.651679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba7846-38ca-4d49-8801-ec351f5f1d93 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:49.661474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1266/2507] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:24:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:24:49.687337 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-97511bc5-cd14-457e-9f47-84977e8d7530 None None] MFA Rules not processed for user `5901fc063f874a6c91cbf9e3b6183944`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:24:49.689079 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-97511bc5-cd14-457e-9f47-84977e8d7530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:49.783925 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1242/2508] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:24:49 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:24:49.785735 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba7846-38ca-4d49-8801-ec351f5f1d93 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:49.787677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-55ba7846-38ca-4d49-8801-ec351f5f1d93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:49.902326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba7846-38ca-4d49-8801-ec351f5f1d93 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:49.906132 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0d049-ee97-4e78-bd25-73685d48c005 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fd398c547554629927d2a80b3a8dee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:49.907003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0d049-ee97-4e78-bd25-73685d48c005 service glance] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:49.907476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0d049-ee97-4e78-bd25-73685d48c005 service glance] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:49.907976 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0d049-ee97-4e78-bd25-73685d48c005 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:49.910395 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-6cb0d049-ee97-4e78-bd25-73685d48c005 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:49.990075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb0d049-ee97-4e78-bd25-73685d48c005 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:49.991524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb0d049-ee97-4e78-bd25-73685d48c005 service glance] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:50.005137 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1267/2509] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:24:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:24:50.076397 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97511bc5-cd14-457e-9f47-84977e8d7530 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:50.086589 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97511bc5-cd14-457e-9f47-84977e8d7530 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:50.090366 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7003e-c506-43bc-bae2-b328e96928b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:50.091020 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7003e-c506-43bc-bae2-b328e96928b8 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:50.091300 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7003e-c506-43bc-bae2-b328e96928b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:50.091571 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7003e-c506-43bc-bae2-b328e96928b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:50.093353 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-64a7003e-c506-43bc-bae2-b328e96928b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:50.103265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7003e-c506-43bc-bae2-b328e96928b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:50.104778 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7003e-c506-43bc-bae2-b328e96928b8 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:50.112874 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1243/2510] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:24:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:24:50.197837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0d049-ee97-4e78-bd25-73685d48c005 service glance] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:50.207192 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0d049-ee97-4e78-bd25-73685d48c005 service glance] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:50.211261 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58910e7a-38a0-47d5-990d-d5ff786c1b51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:50.211902 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58910e7a-38a0-47d5-990d-d5ff786c1b51 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:50.212180 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58910e7a-38a0-47d5-990d-d5ff786c1b51 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:50.212446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58910e7a-38a0-47d5-990d-d5ff786c1b51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:50.214312 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-58910e7a-38a0-47d5-990d-d5ff786c1b51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:50.227935 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58910e7a-38a0-47d5-990d-d5ff786c1b51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:50.229407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58910e7a-38a0-47d5-990d-d5ff786c1b51 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:50.240402 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1268/2511] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:24:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 10:24:50.312702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7003e-c506-43bc-bae2-b328e96928b8 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:50.314163 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-64a7003e-c506-43bc-bae2-b328e96928b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:50.386659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7003e-c506-43bc-bae2-b328e96928b8 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:50.391883 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35482efc-118e-43a2-bc15-d2b78418946b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9e458c2c5f4ed1b3eddd84c2ec1703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:50.392538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35482efc-118e-43a2-bc15-d2b78418946b service placement] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:50.392867 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35482efc-118e-43a2-bc15-d2b78418946b service placement] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:50.393143 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35482efc-118e-43a2-bc15-d2b78418946b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:50.395026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-35482efc-118e-43a2-bc15-d2b78418946b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:50.407260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35482efc-118e-43a2-bc15-d2b78418946b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:50.411007 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35482efc-118e-43a2-bc15-d2b78418946b service placement] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:50.424006 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1244/2512] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:24:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:24:50.916579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58910e7a-38a0-47d5-990d-d5ff786c1b51 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:50.983920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58910e7a-38a0-47d5-990d-d5ff786c1b51 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:50.989146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b8af5-6c31-45d4-b7ad-c3ac4fe7dcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:50.990043 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b8af5-6c31-45d4-b7ad-c3ac4fe7dcf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:50.990318 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b8af5-6c31-45d4-b7ad-c3ac4fe7dcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:50.990604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b8af5-6c31-45d4-b7ad-c3ac4fe7dcf2 admin admin] PATH_INFO: `/v3/users/4f8e978e56a441439cf374d43a02970d` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:50.991796 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b8af5-6c31-45d4-b7ad-c3ac4fe7dcf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f8e978e56a441439cf374d43a02970d)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:50.993825 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b8af5-6c31-45d4-b7ad-c3ac4fe7dcf2 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:51.052918 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-378b8af5-6c31-45d4-b7ad-c3ac4fe7dcf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f8e978e56a441439cf374d43a02970d'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:51.059686 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-378b8af5-6c31-45d4-b7ad-c3ac4fe7dcf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f8e978e56a441439cf374d43a02970d'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:51.060376 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-378b8af5-6c31-45d4-b7ad-c3ac4fe7dcf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f8e978e56a441439cf374d43a02970d'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:51.069628 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-378b8af5-6c31-45d4-b7ad-c3ac4fe7dcf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f8e978e56a441439cf374d43a02970d'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:51.070254 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-378b8af5-6c31-45d4-b7ad-c3ac4fe7dcf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f8e978e56a441439cf374d43a02970d'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:51.089690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1269/2513] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:24:50 2022] DELETE /identity/v3/users/4f8e978e56a441439cf374d43a02970d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:24:51.269165 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35482efc-118e-43a2-bc15-d2b78418946b service placement] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:51.281282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35482efc-118e-43a2-bc15-d2b78418946b service placement] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:51.344990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72edb040-942f-436c-aafc-a52365684c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:51.345872 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edb040-942f-436c-aafc-a52365684c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:51.346142 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edb040-942f-436c-aafc-a52365684c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:51.346412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edb040-942f-436c-aafc-a52365684c71 admin admin] PATH_INFO: `/v3/projects/a7c2e9a8c1204ee5a62a80c6c86ffd78` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:51.347624 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72edb040-942f-436c-aafc-a52365684c71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7c2e9a8c1204ee5a62a80c6c86ffd78)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:51.349687 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72edb040-942f-436c-aafc-a52365684c71 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:51.398837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-72edb040-942f-436c-aafc-a52365684c71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7c2e9a8c1204ee5a62a80c6c86ffd78', 'request_id': 'req-72edb040-942f-436c-aafc-a52365684c71'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:51.404985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-72edb040-942f-436c-aafc-a52365684c71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7c2e9a8c1204ee5a62a80c6c86ffd78', 'request_id': 'req-72edb040-942f-436c-aafc-a52365684c71'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:51.410782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-72edb040-942f-436c-aafc-a52365684c71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7c2e9a8c1204ee5a62a80c6c86ffd78', 'request_id': 'req-72edb040-942f-436c-aafc-a52365684c71'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:51.425658 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-72edb040-942f-436c-aafc-a52365684c71 admin admin] The token cache is being invalidate because project a7c2e9a8c1204ee5a62a80c6c86ffd78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:24:51.425905 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-72edb040-942f-436c-aafc-a52365684c71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:51.426632 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-72edb040-942f-436c-aafc-a52365684c71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:51.428242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1245/2514] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:24:51 2022] DELETE /identity/v3/projects/a7c2e9a8c1204ee5a62a80c6c86ffd78 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:24:51.432260 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b8af5-6c31-45d4-b7ad-c3ac4fe7dcf2 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:51.433819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-378b8af5-6c31-45d4-b7ad-c3ac4fe7dcf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:51.446982 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b8af5-6c31-45d4-b7ad-c3ac4fe7dcf2 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:51.504737 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6623dd-33ed-4dbc-9faa-22ff376fe0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:51.505611 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6623dd-33ed-4dbc-9faa-22ff376fe0c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:51.505881 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6623dd-33ed-4dbc-9faa-22ff376fe0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:51.506190 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6623dd-33ed-4dbc-9faa-22ff376fe0c7 admin admin] PATH_INFO: `/v3/users/5901fc063f874a6c91cbf9e3b6183944` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:51.507656 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6623dd-33ed-4dbc-9faa-22ff376fe0c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5901fc063f874a6c91cbf9e3b6183944)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:51.509697 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6623dd-33ed-4dbc-9faa-22ff376fe0c7 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:51.560085 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bb6623dd-33ed-4dbc-9faa-22ff376fe0c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5901fc063f874a6c91cbf9e3b6183944'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:51.567793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bb6623dd-33ed-4dbc-9faa-22ff376fe0c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5901fc063f874a6c91cbf9e3b6183944'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:51.568694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bb6623dd-33ed-4dbc-9faa-22ff376fe0c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5901fc063f874a6c91cbf9e3b6183944'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:51.578089 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bb6623dd-33ed-4dbc-9faa-22ff376fe0c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5901fc063f874a6c91cbf9e3b6183944'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:51.580406 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bb6623dd-33ed-4dbc-9faa-22ff376fe0c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5901fc063f874a6c91cbf9e3b6183944'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:51.599069 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1270/2515] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:24:51 2022] DELETE /identity/v3/users/5901fc063f874a6c91cbf9e3b6183944 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:24:51.853289 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72edb040-942f-436c-aafc-a52365684c71 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:51.854777 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-72edb040-942f-436c-aafc-a52365684c71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:51.868410 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72edb040-942f-436c-aafc-a52365684c71 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:51.930249 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d564904-6466-46ae-9060-463d63da0b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:51.931600 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d564904-6466-46ae-9060-463d63da0b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:51.931600 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d564904-6466-46ae-9060-463d63da0b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:51.931792 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d564904-6466-46ae-9060-463d63da0b83 admin admin] PATH_INFO: `/v3/projects/e18b57fe7a0f47598dbfcf34c43acc24` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:51.932887 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d564904-6466-46ae-9060-463d63da0b83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e18b57fe7a0f47598dbfcf34c43acc24)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:51.935065 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d564904-6466-46ae-9060-463d63da0b83 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:51.988315 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-8d564904-6466-46ae-9060-463d63da0b83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e18b57fe7a0f47598dbfcf34c43acc24', 'request_id': 'req-8d564904-6466-46ae-9060-463d63da0b83'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:51.994511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-8d564904-6466-46ae-9060-463d63da0b83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e18b57fe7a0f47598dbfcf34c43acc24', 'request_id': 'req-8d564904-6466-46ae-9060-463d63da0b83'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:51.999770 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-8d564904-6466-46ae-9060-463d63da0b83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e18b57fe7a0f47598dbfcf34c43acc24', 'request_id': 'req-8d564904-6466-46ae-9060-463d63da0b83'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:52.013261 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-8d564904-6466-46ae-9060-463d63da0b83 admin admin] The token cache is being invalidate because project e18b57fe7a0f47598dbfcf34c43acc24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:24:52.013647 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-8d564904-6466-46ae-9060-463d63da0b83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:52.014456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-8d564904-6466-46ae-9060-463d63da0b83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:52.016013 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1246/2516] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:24:51 2022] DELETE /identity/v3/projects/e18b57fe7a0f47598dbfcf34c43acc24 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:24:52.020980 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53f58f-9e6e-4de9-b713-0f59bc196c14 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:52.021499 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53f58f-9e6e-4de9-b713-0f59bc196c14 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:52.021499 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53f58f-9e6e-4de9-b713-0f59bc196c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:52.042412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-4e53f58f-9e6e-4de9-b713-0f59bc196c14 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:24:52.044063 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-4e53f58f-9e6e-4de9-b713-0f59bc196c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:52.110245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1271/2517] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 10:24:52.118018 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a7a4b-7e98-4b91-ab20-03ee27f241a9 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:52.118286 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a7a4b-7e98-4b91-ab20-03ee27f241a9 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:52.118569 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a7a4b-7e98-4b91-ab20-03ee27f241a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:52.139751 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a483ce-5ea3-4405-8e4c-a141a135dd76 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:52.139949 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-9e3a7a4b-7e98-4b91-ab20-03ee27f241a9 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:24:52.140061 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a483ce-5ea3-4405-8e4c-a141a135dd76 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:52.140294 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a483ce-5ea3-4405-8e4c-a141a135dd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:52.141465 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-9e3a7a4b-7e98-4b91-ab20-03ee27f241a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:52.153603 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1247/2518] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:24:52.154734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a7a4b-7e98-4b91-ab20-03ee27f241a9 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:52.160640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-38a483ce-5ea3-4405-8e4c-a141a135dd76 None None] MFA Rules not processed for user `af62b2d3f73f482b8db4173af33752c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:24:52.161923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-38a483ce-5ea3-4405-8e4c-a141a135dd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:52.164746 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a7a4b-7e98-4b91-ab20-03ee27f241a9 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:52.170174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f6aa7-fdb3-454c-a58b-4925054add83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:52.170833 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f6aa7-fdb3-454c-a58b-4925054add83 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:52.171110 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f6aa7-fdb3-454c-a58b-4925054add83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:52.171446 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f6aa7-fdb3-454c-a58b-4925054add83 admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:52.172160 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f6aa7-fdb3-454c-a58b-4925054add83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 10:24:52.172463 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f6aa7-fdb3-454c-a58b-4925054add83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:52.174142 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f6aa7-fdb3-454c-a58b-4925054add83 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:52.186461 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1248/2519] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 10:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 10:24:52.190789 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f6aa7-fdb3-454c-a58b-4925054add83 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:52.203389 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f6aa7-fdb3-454c-a58b-4925054add83 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:52.208631 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caae963-d048-43d8-8121-5b2fa98045f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:52.210285 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caae963-d048-43d8-8121-5b2fa98045f3 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:52.210781 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caae963-d048-43d8-8121-5b2fa98045f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:52.211255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caae963-d048-43d8-8121-5b2fa98045f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:52.212165 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caae963-d048-43d8-8121-5b2fa98045f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:52.214298 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caae963-d048-43d8-8121-5b2fa98045f3 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:52.232123 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1272/2520] 10.222.0.26 () {58 vars in 1009 bytes} [Mon May 9 10:24:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 10:24:52.241177 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a483ce-5ea3-4405-8e4c-a141a135dd76 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:52.242861 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-38a483ce-5ea3-4405-8e4c-a141a135dd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:52.251764 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1249/2521] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:24:52 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:24:52.255827 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caae963-d048-43d8-8121-5b2fa98045f3 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:52.317434 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a483ce-5ea3-4405-8e4c-a141a135dd76 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:52.326081 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caae963-d048-43d8-8121-5b2fa98045f3 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:52.330860 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee78704f-a6f8-482d-9583-b5cd12f7fed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:52.331526 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78704f-a6f8-482d-9583-b5cd12f7fed9 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:52.331802 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78704f-a6f8-482d-9583-b5cd12f7fed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:52.332290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78704f-a6f8-482d-9583-b5cd12f7fed9 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:52.333238 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee78704f-a6f8-482d-9583-b5cd12f7fed9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:52.335761 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee78704f-a6f8-482d-9583-b5cd12f7fed9 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:52.358729 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1250/2522] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:24:52 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:24:52.362963 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee78704f-a6f8-482d-9583-b5cd12f7fed9 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:52.374210 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee78704f-a6f8-482d-9583-b5cd12f7fed9 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:52.381229 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e95ad-1f54-42da-9026-c5d74f78599c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:52.381909 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e95ad-1f54-42da-9026-c5d74f78599c admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:52.382237 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e95ad-1f54-42da-9026-c5d74f78599c admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:52.382685 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e95ad-1f54-42da-9026-c5d74f78599c admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:52.383911 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e95ad-1f54-42da-9026-c5d74f78599c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:52.385579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e95ad-1f54-42da-9026-c5d74f78599c admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:52.389559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d616d-09aa-40d4-ba2e-251ba7df301c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380e04358f2349aa912d6a97518877cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:52.390242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d616d-09aa-40d4-ba2e-251ba7df301c service swift] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:52.390503 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d616d-09aa-40d4-ba2e-251ba7df301c service swift] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:52.390778 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d616d-09aa-40d4-ba2e-251ba7df301c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:52.397087 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1251/2523] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 10:24:52.401033 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e95ad-1f54-42da-9026-c5d74f78599c admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:52.411716 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e95ad-1f54-42da-9026-c5d74f78599c admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:52.417411 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174be9af-a74d-4f7a-9b07-efca24777cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:52.418208 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174be9af-a74d-4f7a-9b07-efca24777cce admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:52.418469 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174be9af-a74d-4f7a-9b07-efca24777cce admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:52.418676 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174be9af-a74d-4f7a-9b07-efca24777cce admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:52.419413 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174be9af-a74d-4f7a-9b07-efca24777cce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:52.421072 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174be9af-a74d-4f7a-9b07-efca24777cce admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:52.432580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1252/2524] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 10:24:52.436391 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174be9af-a74d-4f7a-9b07-efca24777cce admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:52.448548 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174be9af-a74d-4f7a-9b07-efca24777cce admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:52.448919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d616d-09aa-40d4-ba2e-251ba7df301c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:52.450627 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d616d-09aa-40d4-ba2e-251ba7df301c service swift] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:52.455416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb76a7-abc3-42af-a14e-dc0991e6767f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:52.456305 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb76a7-abc3-42af-a14e-dc0991e6767f admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:52.456584 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb76a7-abc3-42af-a14e-dc0991e6767f admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:52.456903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb76a7-abc3-42af-a14e-dc0991e6767f admin admin] PATH_INFO: `/v3/projects/895c158217d44d4b90c1ffdb77e3bcc1/users/94b5ecb511cb4ed38f816de27b480688/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:52.459644 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1273/2525] 10.222.0.26 () {60 vars in 1384 bytes} [Mon May 9 10:24:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 10:24:52.476643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb76a7-abc3-42af-a14e-dc0991e6767f admin admin] RBAC: Authorizing `identity:create_grant(project_id=895c158217d44d4b90c1ffdb77e3bcc1, user_id=94b5ecb511cb4ed38f816de27b480688, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:52.479438 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb76a7-abc3-42af-a14e-dc0991e6767f admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:52.493226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1253/2526] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:24:52 2022] PUT /identity/v3/projects/895c158217d44d4b90c1ffdb77e3bcc1/users/94b5ecb511cb4ed38f816de27b480688/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:24:52.503972 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd98f2-a404-48ff-8f00-f831edcdbb0c None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:52.504292 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd98f2-a404-48ff-8f00-f831edcdbb0c None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:52.504564 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd98f2-a404-48ff-8f00-f831edcdbb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:52.532200 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-33dd98f2-a404-48ff-8f00-f831edcdbb0c None None] MFA Rules not processed for user `94b5ecb511cb4ed38f816de27b480688`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:24:52.533562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-33dd98f2-a404-48ff-8f00-f831edcdbb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:52.618926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1274/2527] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:24:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 10:24:52.620331 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb76a7-abc3-42af-a14e-dc0991e6767f admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:52.734611 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb76a7-abc3-42af-a14e-dc0991e6767f admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:52.743490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a54977-05a8-4293-be00-44ad5622fdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:52.745216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a54977-05a8-4293-be00-44ad5622fdfb admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:52.746216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a54977-05a8-4293-be00-44ad5622fdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:52.746768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a54977-05a8-4293-be00-44ad5622fdfb admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:52.747782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a54977-05a8-4293-be00-44ad5622fdfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:52.752050 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a54977-05a8-4293-be00-44ad5622fdfb admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:52.788417 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1254/2528] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:24:52 2022] POST /identity/v3/projects => generated 371 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:24:52.789698 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd98f2-a404-48ff-8f00-f831edcdbb0c None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:52.813042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd98f2-a404-48ff-8f00-f831edcdbb0c None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:52.892932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4873d-1df7-46f3-a61b-c31ef1bd1a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:52.893627 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4873d-1df7-46f3-a61b-c31ef1bd1a61 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:52.894055 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4873d-1df7-46f3-a61b-c31ef1bd1a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:52.894379 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4873d-1df7-46f3-a61b-c31ef1bd1a61 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:52.895284 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef4873d-1df7-46f3-a61b-c31ef1bd1a61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:52.897576 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef4873d-1df7-46f3-a61b-c31ef1bd1a61 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:52.921413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1275/2529] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:24:52 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:24:52.925293 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a54977-05a8-4293-be00-44ad5622fdfb admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:52.944998 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a54977-05a8-4293-be00-44ad5622fdfb admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:52.950928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5b684-fee0-437b-beaa-b34ba4033d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:52.951807 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5b684-fee0-437b-beaa-b34ba4033d43 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:52.952092 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5b684-fee0-437b-beaa-b34ba4033d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:52.952557 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5b684-fee0-437b-beaa-b34ba4033d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:52.953342 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba5b684-fee0-437b-beaa-b34ba4033d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:52.955025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba5b684-fee0-437b-beaa-b34ba4033d43 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:52.974226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1255/2530] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 10:24:52.975450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4873d-1df7-46f3-a61b-c31ef1bd1a61 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:52.985783 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4873d-1df7-46f3-a61b-c31ef1bd1a61 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:52.990818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a51b7-c296-41b1-9481-1b79f43c0201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:52.991721 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a51b7-c296-41b1-9481-1b79f43c0201 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:52.991995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a51b7-c296-41b1-9481-1b79f43c0201 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:52.992262 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a51b7-c296-41b1-9481-1b79f43c0201 admin admin] PATH_INFO: `/v3/projects/5bace4e96a1f4e3e9d913706529ef2b8/users/9825925e9ebb483a8a1334674b3fb70b/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:53.010326 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a51b7-c296-41b1-9481-1b79f43c0201 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bace4e96a1f4e3e9d913706529ef2b8, user_id=9825925e9ebb483a8a1334674b3fb70b, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:53.013850 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a51b7-c296-41b1-9481-1b79f43c0201 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:53.034348 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5b684-fee0-437b-beaa-b34ba4033d43 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:53.034781 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1276/2531] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:24:52 2022] PUT /identity/v3/projects/5bace4e96a1f4e3e9d913706529ef2b8/users/9825925e9ebb483a8a1334674b3fb70b/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:24:53.045255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5b684-fee0-437b-beaa-b34ba4033d43 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:53.110116 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c284a-09f7-401d-9151-54ca8897059e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:53.111300 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c284a-09f7-401d-9151-54ca8897059e admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:53.111740 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c284a-09f7-401d-9151-54ca8897059e admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:53.112174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c284a-09f7-401d-9151-54ca8897059e admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:53.113107 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c284a-09f7-401d-9151-54ca8897059e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:53.114823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c284a-09f7-401d-9151-54ca8897059e admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:53.128260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1256/2532] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:24:53.132923 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a51b7-c296-41b1-9481-1b79f43c0201 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:53.143368 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a51b7-c296-41b1-9481-1b79f43c0201 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:53.148385 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11a7ae-ac2c-4e37-b893-0aef0cb41f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:53.149538 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11a7ae-ac2c-4e37-b893-0aef0cb41f8a admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:53.149980 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11a7ae-ac2c-4e37-b893-0aef0cb41f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:53.150433 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11a7ae-ac2c-4e37-b893-0aef0cb41f8a admin admin] PATH_INFO: `/v3/projects/5bace4e96a1f4e3e9d913706529ef2b8/users/9825925e9ebb483a8a1334674b3fb70b/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:53.152785 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc11a7ae-ac2c-4e37-b893-0aef0cb41f8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bace4e96a1f4e3e9d913706529ef2b8, user_id=9825925e9ebb483a8a1334674b3fb70b, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:53.155666 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc11a7ae-ac2c-4e37-b893-0aef0cb41f8a admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:53.166453 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1277/2533] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:24:53 2022] PUT /identity/v3/projects/5bace4e96a1f4e3e9d913706529ef2b8/users/9825925e9ebb483a8a1334674b3fb70b/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:24:53.177976 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34964e6-690d-47aa-baa5-5995f8a1caef None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:53.178297 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34964e6-690d-47aa-baa5-5995f8a1caef None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:53.178577 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34964e6-690d-47aa-baa5-5995f8a1caef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:53.206989 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-c34964e6-690d-47aa-baa5-5995f8a1caef None None] MFA Rules not processed for user `9825925e9ebb483a8a1334674b3fb70b`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:24:53.208247 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-c34964e6-690d-47aa-baa5-5995f8a1caef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:53.289531 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1257/2534] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:24:53 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:24:53.301315 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11a7ae-ac2c-4e37-b893-0aef0cb41f8a admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:53.303109 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-cc11a7ae-ac2c-4e37-b893-0aef0cb41f8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:53.361562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11a7ae-ac2c-4e37-b893-0aef0cb41f8a admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:53.366396 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28ffc88-01c1-4976-be48-bdfbdcdf0ee1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:53.367503 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ffc88-01c1-4976-be48-bdfbdcdf0ee1 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:53.367618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ffc88-01c1-4976-be48-bdfbdcdf0ee1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:53.367828 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ffc88-01c1-4976-be48-bdfbdcdf0ee1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:53.377365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28ffc88-01c1-4976-be48-bdfbdcdf0ee1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:53.378807 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28ffc88-01c1-4976-be48-bdfbdcdf0ee1 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:53.389842 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1278/2535] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:24:53 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 10:24:53.822784 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34964e6-690d-47aa-baa5-5995f8a1caef None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:53.887396 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34964e6-690d-47aa-baa5-5995f8a1caef None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:53.892354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5d5fb-1317-4850-8cf0-efd9d6681fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:53.893248 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5d5fb-1317-4850-8cf0-efd9d6681fc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:53.893525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5d5fb-1317-4850-8cf0-efd9d6681fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:53.893797 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5d5fb-1317-4850-8cf0-efd9d6681fc2 admin admin] PATH_INFO: `/v3/users/94b5ecb511cb4ed38f816de27b480688` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:53.895050 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5d5fb-1317-4850-8cf0-efd9d6681fc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94b5ecb511cb4ed38f816de27b480688)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:53.896998 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5d5fb-1317-4850-8cf0-efd9d6681fc2 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:53.948245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-89c5d5fb-1317-4850-8cf0-efd9d6681fc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94b5ecb511cb4ed38f816de27b480688'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:53.954566 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-89c5d5fb-1317-4850-8cf0-efd9d6681fc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94b5ecb511cb4ed38f816de27b480688'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:53.963666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-89c5d5fb-1317-4850-8cf0-efd9d6681fc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94b5ecb511cb4ed38f816de27b480688'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:53.964280 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-89c5d5fb-1317-4850-8cf0-efd9d6681fc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94b5ecb511cb4ed38f816de27b480688'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:53.964915 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-89c5d5fb-1317-4850-8cf0-efd9d6681fc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94b5ecb511cb4ed38f816de27b480688'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:53.988121 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1258/2536] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:24:53 2022] DELETE /identity/v3/users/94b5ecb511cb4ed38f816de27b480688 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:24:53.993162 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4f834-a520-4949-a7bc-d37e30f4d56b None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:53.993444 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4f834-a520-4949-a7bc-d37e30f4d56b None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:53.993710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4f834-a520-4949-a7bc-d37e30f4d56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:54.018013 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-55a4f834-a520-4949-a7bc-d37e30f4d56b None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:24:54.019228 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-55a4f834-a520-4949-a7bc-d37e30f4d56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:54.077295 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1279/2537] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:24:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:24:54.087569 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5d5fb-1317-4850-8cf0-efd9d6681fc2 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:54.089108 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-89c5d5fb-1317-4850-8cf0-efd9d6681fc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:54.164449 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5d5fb-1317-4850-8cf0-efd9d6681fc2 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:54.168533 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f48715-4286-4ecb-b838-31183f031f3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:54.169205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f48715-4286-4ecb-b838-31183f031f3c service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:54.169482 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f48715-4286-4ecb-b838-31183f031f3c service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:54.169745 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f48715-4286-4ecb-b838-31183f031f3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:54.180320 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f48715-4286-4ecb-b838-31183f031f3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:54.181768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f48715-4286-4ecb-b838-31183f031f3c service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:54.191349 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1259/2538] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:24:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:24:54.505110 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4f834-a520-4949-a7bc-d37e30f4d56b None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:54.506603 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-55a4f834-a520-4949-a7bc-d37e30f4d56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:54.528634 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4f834-a520-4949-a7bc-d37e30f4d56b None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:54.548367 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8369fa56-73fb-4aaf-95b4-1aedce09ffaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:54.550907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369fa56-73fb-4aaf-95b4-1aedce09ffaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:54.551211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369fa56-73fb-4aaf-95b4-1aedce09ffaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:54.551489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369fa56-73fb-4aaf-95b4-1aedce09ffaf admin admin] PATH_INFO: `/v3/projects/895c158217d44d4b90c1ffdb77e3bcc1` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:54.552870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8369fa56-73fb-4aaf-95b4-1aedce09ffaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=895c158217d44d4b90c1ffdb77e3bcc1)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:54.554826 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8369fa56-73fb-4aaf-95b4-1aedce09ffaf admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:54.622677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8369fa56-73fb-4aaf-95b4-1aedce09ffaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '895c158217d44d4b90c1ffdb77e3bcc1', 'request_id': 'req-8369fa56-73fb-4aaf-95b4-1aedce09ffaf'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:54.630651 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8369fa56-73fb-4aaf-95b4-1aedce09ffaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '895c158217d44d4b90c1ffdb77e3bcc1', 'request_id': 'req-8369fa56-73fb-4aaf-95b4-1aedce09ffaf'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:54.637784 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8369fa56-73fb-4aaf-95b4-1aedce09ffaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '895c158217d44d4b90c1ffdb77e3bcc1', 'request_id': 'req-8369fa56-73fb-4aaf-95b4-1aedce09ffaf'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:54.654591 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8369fa56-73fb-4aaf-95b4-1aedce09ffaf admin admin] The token cache is being invalidate because project 895c158217d44d4b90c1ffdb77e3bcc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:24:54.654968 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8369fa56-73fb-4aaf-95b4-1aedce09ffaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:54.655671 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8369fa56-73fb-4aaf-95b4-1aedce09ffaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:54.660727 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1280/2539] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:24:54 2022] DELETE /identity/v3/projects/895c158217d44d4b90c1ffdb77e3bcc1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:24:54.662200 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f48715-4286-4ecb-b838-31183f031f3c service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:54.673217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f48715-4286-4ecb-b838-31183f031f3c service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:54.756421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7728b-f011-4d35-a730-1fd894eb6da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:54.757239 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7728b-f011-4d35-a730-1fd894eb6da0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:54.757515 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7728b-f011-4d35-a730-1fd894eb6da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:54.757786 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7728b-f011-4d35-a730-1fd894eb6da0 admin admin] PATH_INFO: `/v3/users/9825925e9ebb483a8a1334674b3fb70b` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:54.759338 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb7728b-f011-4d35-a730-1fd894eb6da0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9825925e9ebb483a8a1334674b3fb70b)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:54.761307 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb7728b-f011-4d35-a730-1fd894eb6da0 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:54.828475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6eb7728b-f011-4d35-a730-1fd894eb6da0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9825925e9ebb483a8a1334674b3fb70b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:54.837224 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6eb7728b-f011-4d35-a730-1fd894eb6da0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9825925e9ebb483a8a1334674b3fb70b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:54.846702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6eb7728b-f011-4d35-a730-1fd894eb6da0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9825925e9ebb483a8a1334674b3fb70b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:54.847998 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6eb7728b-f011-4d35-a730-1fd894eb6da0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9825925e9ebb483a8a1334674b3fb70b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:54.848995 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6eb7728b-f011-4d35-a730-1fd894eb6da0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9825925e9ebb483a8a1334674b3fb70b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:54.873407 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1260/2540] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:24:54 2022] DELETE /identity/v3/users/9825925e9ebb483a8a1334674b3fb70b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:24:55.188078 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8369fa56-73fb-4aaf-95b4-1aedce09ffaf admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:55.189542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-8369fa56-73fb-4aaf-95b4-1aedce09ffaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:55.204423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8369fa56-73fb-4aaf-95b4-1aedce09ffaf admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:55.265894 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f482374-be77-46e7-87cb-eaf2fd462a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:55.266767 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f482374-be77-46e7-87cb-eaf2fd462a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:55.267044 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f482374-be77-46e7-87cb-eaf2fd462a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:55.267317 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f482374-be77-46e7-87cb-eaf2fd462a79 admin admin] PATH_INFO: `/v3/projects/5bace4e96a1f4e3e9d913706529ef2b8` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:55.268493 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f482374-be77-46e7-87cb-eaf2fd462a79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bace4e96a1f4e3e9d913706529ef2b8)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:55.270578 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f482374-be77-46e7-87cb-eaf2fd462a79 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:55.317159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-3f482374-be77-46e7-87cb-eaf2fd462a79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bace4e96a1f4e3e9d913706529ef2b8', 'request_id': 'req-3f482374-be77-46e7-87cb-eaf2fd462a79'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:55.323271 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-3f482374-be77-46e7-87cb-eaf2fd462a79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bace4e96a1f4e3e9d913706529ef2b8', 'request_id': 'req-3f482374-be77-46e7-87cb-eaf2fd462a79'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:55.328516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-3f482374-be77-46e7-87cb-eaf2fd462a79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bace4e96a1f4e3e9d913706529ef2b8', 'request_id': 'req-3f482374-be77-46e7-87cb-eaf2fd462a79'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:55.343629 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-3f482374-be77-46e7-87cb-eaf2fd462a79 admin admin] The token cache is being invalidate because project 5bace4e96a1f4e3e9d913706529ef2b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:24:55.344138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-3f482374-be77-46e7-87cb-eaf2fd462a79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:55.344870 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-3f482374-be77-46e7-87cb-eaf2fd462a79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:24:55.350183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1281/2541] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:24:55 2022] DELETE /identity/v3/projects/5bace4e96a1f4e3e9d913706529ef2b8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:24:55.351828 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b0431-6426-44d5-a7a2-d7b21123960b None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:55.352322 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b0431-6426-44d5-a7a2-d7b21123960b None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:55.352598 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b0431-6426-44d5-a7a2-d7b21123960b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:55.373346 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-566b0431-6426-44d5-a7a2-d7b21123960b None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:24:55.374809 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-566b0431-6426-44d5-a7a2-d7b21123960b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:55.447665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1261/2542] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:24:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:24:55.455685 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191d246-4061-4171-937b-caa50df91d77 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:55.455959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191d246-4061-4171-937b-caa50df91d77 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:55.456276 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191d246-4061-4171-937b-caa50df91d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:55.479018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-1191d246-4061-4171-937b-caa50df91d77 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:24:55.480717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-1191d246-4061-4171-937b-caa50df91d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:55.492221 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1282/2543] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:24:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:24:55.496581 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b0431-6426-44d5-a7a2-d7b21123960b None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:55.507140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b0431-6426-44d5-a7a2-d7b21123960b None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:55.513085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0a331-a509-48fa-a97d-3ec0281526c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:55.513897 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0a331-a509-48fa-a97d-3ec0281526c2 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:55.514252 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0a331-a509-48fa-a97d-3ec0281526c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:55.514526 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0a331-a509-48fa-a97d-3ec0281526c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:55.515290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0a331-a509-48fa-a97d-3ec0281526c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 10:24:55.515584 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0a331-a509-48fa-a97d-3ec0281526c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:55.517263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0a331-a509-48fa-a97d-3ec0281526c2 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:55.530878 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1262/2544] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 10:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 10:24:55.535293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191d246-4061-4171-937b-caa50df91d77 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:55.545648 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191d246-4061-4171-937b-caa50df91d77 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:55.550651 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819af90-3d5a-4f12-abbe-95c27cc56753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:55.551383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819af90-3d5a-4f12-abbe-95c27cc56753 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:55.551661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819af90-3d5a-4f12-abbe-95c27cc56753 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:55.551933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819af90-3d5a-4f12-abbe-95c27cc56753 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:55.552632 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6819af90-3d5a-4f12-abbe-95c27cc56753 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:55.554582 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6819af90-3d5a-4f12-abbe-95c27cc56753 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:55.581822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1283/2545] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:24:55 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:24:55.586264 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0a331-a509-48fa-a97d-3ec0281526c2 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:55.606216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0a331-a509-48fa-a97d-3ec0281526c2 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:55.671706 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac91e7-49c9-44ac-91d9-595aeef50568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:55.690885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac91e7-49c9-44ac-91d9-595aeef50568 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:55.690885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac91e7-49c9-44ac-91d9-595aeef50568 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:55.690885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac91e7-49c9-44ac-91d9-595aeef50568 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:55.690885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac91e7-49c9-44ac-91d9-595aeef50568 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:55.690885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac91e7-49c9-44ac-91d9-595aeef50568 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:55.718086 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1263/2546] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:24:55 2022] POST /identity/v3/users => generated 283 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:24:55.723631 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819af90-3d5a-4f12-abbe-95c27cc56753 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:55.741264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819af90-3d5a-4f12-abbe-95c27cc56753 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:55.746751 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33257533-f8f7-4e7f-a861-cc61a706c08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:55.747437 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33257533-f8f7-4e7f-a861-cc61a706c08e admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:55.747742 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33257533-f8f7-4e7f-a861-cc61a706c08e admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:55.748012 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33257533-f8f7-4e7f-a861-cc61a706c08e admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:55.748808 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33257533-f8f7-4e7f-a861-cc61a706c08e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:55.750468 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33257533-f8f7-4e7f-a861-cc61a706c08e admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:55.761302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1284/2547] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:24:55.765387 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac91e7-49c9-44ac-91d9-595aeef50568 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:55.775650 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac91e7-49c9-44ac-91d9-595aeef50568 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:55.780379 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49510d-4c2d-4dc3-a141-70504468a241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:55.781083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49510d-4c2d-4dc3-a141-70504468a241 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:55.781357 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49510d-4c2d-4dc3-a141-70504468a241 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:55.781619 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49510d-4c2d-4dc3-a141-70504468a241 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:55.782342 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49510d-4c2d-4dc3-a141-70504468a241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:55.783954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49510d-4c2d-4dc3-a141-70504468a241 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:55.800041 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1264/2548] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:24:55.801621 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33257533-f8f7-4e7f-a861-cc61a706c08e admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:55.813821 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33257533-f8f7-4e7f-a861-cc61a706c08e admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:55.827028 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61f89f-d522-4195-b5b7-19dc84fba013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:55.831612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61f89f-d522-4195-b5b7-19dc84fba013 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:55.831612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61f89f-d522-4195-b5b7-19dc84fba013 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:55.831612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61f89f-d522-4195-b5b7-19dc84fba013 admin admin] PATH_INFO: `/v3/projects/0e513bbf0edc4facaba307fa6116062d/users/5c5c7e700d50400683e44111f1b76b48/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:55.867218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61f89f-d522-4195-b5b7-19dc84fba013 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e513bbf0edc4facaba307fa6116062d, user_id=5c5c7e700d50400683e44111f1b76b48, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:55.870090 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61f89f-d522-4195-b5b7-19dc84fba013 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:55.896220 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1285/2549] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:24:55 2022] PUT /identity/v3/projects/0e513bbf0edc4facaba307fa6116062d/users/5c5c7e700d50400683e44111f1b76b48/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 10:24:55.906171 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fbcc8-b9a1-40db-aef8-568f6c197207 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:55.906743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fbcc8-b9a1-40db-aef8-568f6c197207 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:55.906743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fbcc8-b9a1-40db-aef8-568f6c197207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:55.962698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-171fbcc8-b9a1-40db-aef8-568f6c197207 None None] MFA Rules not processed for user `5c5c7e700d50400683e44111f1b76b48`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:24:55.968226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-171fbcc8-b9a1-40db-aef8-568f6c197207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:56.049078 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1265/2550] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:24:55 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:24:56.054162 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61f89f-d522-4195-b5b7-19dc84fba013 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:56.120569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61f89f-d522-4195-b5b7-19dc84fba013 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:56.125248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5740515-dad5-45ec-9c28-064b804425f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:56.125917 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5740515-dad5-45ec-9c28-064b804425f2 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:56.126193 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5740515-dad5-45ec-9c28-064b804425f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:56.126461 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5740515-dad5-45ec-9c28-064b804425f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:56.127169 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5740515-dad5-45ec-9c28-064b804425f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:56.129404 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5740515-dad5-45ec-9c28-064b804425f2 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:56.160722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1286/2551] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:24:56 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:24:56.163006 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171fbcc8-b9a1-40db-aef8-568f6c197207 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:56.173602 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171fbcc8-b9a1-40db-aef8-568f6c197207 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:56.230737 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c82dd-d2e1-4a6d-8ad5-922234225a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:56.231415 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c82dd-d2e1-4a6d-8ad5-922234225a66 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:56.231694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c82dd-d2e1-4a6d-8ad5-922234225a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:56.231967 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c82dd-d2e1-4a6d-8ad5-922234225a66 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:56.232684 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c82dd-d2e1-4a6d-8ad5-922234225a66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:56.234650 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c82dd-d2e1-4a6d-8ad5-922234225a66 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:56.258203 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1266/2552] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:24:56 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:24:56.263557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5740515-dad5-45ec-9c28-064b804425f2 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:56.276967 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5740515-dad5-45ec-9c28-064b804425f2 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:56.283172 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03763d-670d-4d46-b664-4befdb151b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:56.284099 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03763d-670d-4d46-b664-4befdb151b1e admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:56.284816 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03763d-670d-4d46-b664-4befdb151b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:56.285465 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03763d-670d-4d46-b664-4befdb151b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:56.286550 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03763d-670d-4d46-b664-4befdb151b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:56.288641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03763d-670d-4d46-b664-4befdb151b1e admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:56.300615 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1287/2553] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:24:56.305770 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c82dd-d2e1-4a6d-8ad5-922234225a66 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:56.319738 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c82dd-d2e1-4a6d-8ad5-922234225a66 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:56.325892 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c6a26-da13-4d34-bb9a-ce09d70a4fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:56.326799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c6a26-da13-4d34-bb9a-ce09d70a4fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:56.327073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c6a26-da13-4d34-bb9a-ce09d70a4fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:56.327344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c6a26-da13-4d34-bb9a-ce09d70a4fb0 admin admin] PATH_INFO: `/v3/projects/7df486e1b37a4d64ad5bd215c0c7f363/users/919111f7a72f4e5d8e77bac3696aae48/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:56.346566 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c6a26-da13-4d34-bb9a-ce09d70a4fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7df486e1b37a4d64ad5bd215c0c7f363, user_id=919111f7a72f4e5d8e77bac3696aae48, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:56.349711 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c6a26-da13-4d34-bb9a-ce09d70a4fb0 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:56.365429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1267/2554] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:24:56 2022] PUT /identity/v3/projects/7df486e1b37a4d64ad5bd215c0c7f363/users/919111f7a72f4e5d8e77bac3696aae48/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 10:24:56.369513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03763d-670d-4d46-b664-4befdb151b1e admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:56.379312 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03763d-670d-4d46-b664-4befdb151b1e admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:56.441569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68127c79-8f6c-4639-8b7b-7eef8786cba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:56.442793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68127c79-8f6c-4639-8b7b-7eef8786cba6 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:56.443077 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68127c79-8f6c-4639-8b7b-7eef8786cba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:56.443353 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68127c79-8f6c-4639-8b7b-7eef8786cba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:56.444823 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68127c79-8f6c-4639-8b7b-7eef8786cba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:56.445956 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68127c79-8f6c-4639-8b7b-7eef8786cba6 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:56.461675 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1288/2555] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:24:56.463783 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c6a26-da13-4d34-bb9a-ce09d70a4fb0 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:56.479177 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c6a26-da13-4d34-bb9a-ce09d70a4fb0 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:56.484191 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c8194-c3e7-4123-aee6-90fc140045f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:56.485385 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c8194-c3e7-4123-aee6-90fc140045f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:56.485727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c8194-c3e7-4123-aee6-90fc140045f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:56.486059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c8194-c3e7-4123-aee6-90fc140045f0 admin admin] PATH_INFO: `/v3/projects/7df486e1b37a4d64ad5bd215c0c7f363/users/919111f7a72f4e5d8e77bac3696aae48/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:56.488580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c8194-c3e7-4123-aee6-90fc140045f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7df486e1b37a4d64ad5bd215c0c7f363, user_id=919111f7a72f4e5d8e77bac3696aae48, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:56.491756 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c8194-c3e7-4123-aee6-90fc140045f0 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:56.504163 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1268/2556] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:24:56 2022] PUT /identity/v3/projects/7df486e1b37a4d64ad5bd215c0c7f363/users/919111f7a72f4e5d8e77bac3696aae48/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:24:56.515561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803fcd52-a173-4c2b-a9d9-7b8fb62134c8 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:56.516130 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803fcd52-a173-4c2b-a9d9-7b8fb62134c8 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:56.516538 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803fcd52-a173-4c2b-a9d9-7b8fb62134c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:56.545879 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-803fcd52-a173-4c2b-a9d9-7b8fb62134c8 None None] MFA Rules not processed for user `919111f7a72f4e5d8e77bac3696aae48`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:24:56.547417 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-803fcd52-a173-4c2b-a9d9-7b8fb62134c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:56.622342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1289/2557] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:24:56 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:24:56.634075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c8194-c3e7-4123-aee6-90fc140045f0 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:56.636048 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-1f7c8194-c3e7-4123-aee6-90fc140045f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:24:56.697200 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c8194-c3e7-4123-aee6-90fc140045f0 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:56.701361 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e6d21-18a6-4c8c-9fe7-b61d04536d71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:56.702015 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e6d21-18a6-4c8c-9fe7-b61d04536d71 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:56.702288 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e6d21-18a6-4c8c-9fe7-b61d04536d71 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:56.702553 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e6d21-18a6-4c8c-9fe7-b61d04536d71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:56.713478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e6d21-18a6-4c8c-9fe7-b61d04536d71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:56.714935 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e6d21-18a6-4c8c-9fe7-b61d04536d71 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:56.726310 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1269/2558] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:24:56 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:24:56.804404 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803fcd52-a173-4c2b-a9d9-7b8fb62134c8 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:24:56.814882 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803fcd52-a173-4c2b-a9d9-7b8fb62134c8 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:24:56.819260 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbaa98-b62c-4e6c-b183-e3486fd4ba62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:24:56.819922 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbaa98-b62c-4e6c-b183-e3486fd4ba62 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:24:56.820338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbaa98-b62c-4e6c-b183-e3486fd4ba62 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:24:56.820613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbaa98-b62c-4e6c-b183-e3486fd4ba62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:24:56.885636 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cbaa98-b62c-4e6c-b183-e3486fd4ba62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:24:56.887377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cbaa98-b62c-4e6c-b183-e3486fd4ba62 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:24:56.898663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1290/2559] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:24:56 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:25:09.026188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e6d21-18a6-4c8c-9fe7-b61d04536d71 service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:09.027959 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-026e6d21-18a6-4c8c-9fe7-b61d04536d71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:09.124621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e6d21-18a6-4c8c-9fe7-b61d04536d71 service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:09.131370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24f929-fc15-4535-b2bd-bcb7dd8860a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:09.132037 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24f929-fc15-4535-b2bd-bcb7dd8860a6 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:09.132316 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24f929-fc15-4535-b2bd-bcb7dd8860a6 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:09.132590 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24f929-fc15-4535-b2bd-bcb7dd8860a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:09.134493 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-2b24f929-fc15-4535-b2bd-bcb7dd8860a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:09.195137 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b24f929-fc15-4535-b2bd-bcb7dd8860a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:09.196794 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b24f929-fc15-4535-b2bd-bcb7dd8860a6 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:09.206550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1270/2560] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:25:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:25:09.257850 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbaa98-b62c-4e6c-b183-e3486fd4ba62 service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:09.259360 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c6cbaa98-b62c-4e6c-b183-e3486fd4ba62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:09.318036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbaa98-b62c-4e6c-b183-e3486fd4ba62 service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:09.322152 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81188654-0b4f-4310-b842-d2179db1b216 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:09.322792 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81188654-0b4f-4310-b842-d2179db1b216 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:09.323057 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81188654-0b4f-4310-b842-d2179db1b216 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:09.323323 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81188654-0b4f-4310-b842-d2179db1b216 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:09.325363 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-81188654-0b4f-4310-b842-d2179db1b216 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:09.341336 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81188654-0b4f-4310-b842-d2179db1b216 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:09.342782 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81188654-0b4f-4310-b842-d2179db1b216 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:09.354071 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1291/2561] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:25:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:25:10.552110 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24f929-fc15-4535-b2bd-bcb7dd8860a6 service nova] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:10.553790 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-2b24f929-fc15-4535-b2bd-bcb7dd8860a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:10.566393 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24f929-fc15-4535-b2bd-bcb7dd8860a6 service nova] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:10.570949 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711681c3-3d3e-459f-a996-0fd25c57bdca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:10.571603 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711681c3-3d3e-459f-a996-0fd25c57bdca service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:10.571880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711681c3-3d3e-459f-a996-0fd25c57bdca service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:10.572145 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711681c3-3d3e-459f-a996-0fd25c57bdca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:10.574299 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-711681c3-3d3e-459f-a996-0fd25c57bdca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:10.590204 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711681c3-3d3e-459f-a996-0fd25c57bdca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:10.591636 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711681c3-3d3e-459f-a996-0fd25c57bdca service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:10.603845 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1271/2562] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:25:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:25:16.085296 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81188654-0b4f-4310-b842-d2179db1b216 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:16.146929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81188654-0b4f-4310-b842-d2179db1b216 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:16.152695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f7abf-5957-48d1-a86f-0ea9b4972320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:16.153571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f7abf-5957-48d1-a86f-0ea9b4972320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:16.153848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f7abf-5957-48d1-a86f-0ea9b4972320 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:16.154114 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f7abf-5957-48d1-a86f-0ea9b4972320 admin admin] PATH_INFO: `/v3/users/5c5c7e700d50400683e44111f1b76b48` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:16.155390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f7abf-5957-48d1-a86f-0ea9b4972320 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c5c7e700d50400683e44111f1b76b48)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:16.157342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f7abf-5957-48d1-a86f-0ea9b4972320 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:16.207563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-7a7f7abf-5957-48d1-a86f-0ea9b4972320 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c5c7e700d50400683e44111f1b76b48'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:16.217342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-7a7f7abf-5957-48d1-a86f-0ea9b4972320 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c5c7e700d50400683e44111f1b76b48'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:16.218563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-7a7f7abf-5957-48d1-a86f-0ea9b4972320 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c5c7e700d50400683e44111f1b76b48'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:16.229122 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-7a7f7abf-5957-48d1-a86f-0ea9b4972320 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c5c7e700d50400683e44111f1b76b48'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:16.229797 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-7a7f7abf-5957-48d1-a86f-0ea9b4972320 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c5c7e700d50400683e44111f1b76b48'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:16.249229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1292/2563] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:25:16 2022] DELETE /identity/v3/users/5c5c7e700d50400683e44111f1b76b48 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:25:16.255138 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e76c3-54cc-4f77-a1d9-0427b98c3914 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:16.255656 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e76c3-54cc-4f77-a1d9-0427b98c3914 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:16.255656 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e76c3-54cc-4f77-a1d9-0427b98c3914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:16.272705 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-5e9e76c3-54cc-4f77-a1d9-0427b98c3914 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:25:16.273915 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-5e9e76c3-54cc-4f77-a1d9-0427b98c3914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:16.333953 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1272/2564] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:25:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:25:16.347299 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f7abf-5957-48d1-a86f-0ea9b4972320 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:16.349020 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-7a7f7abf-5957-48d1-a86f-0ea9b4972320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:16.407457 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f7abf-5957-48d1-a86f-0ea9b4972320 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:16.412263 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31453a5-3164-4f52-8ecd-ebb55153fc12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:16.412943 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31453a5-3164-4f52-8ecd-ebb55153fc12 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:16.413214 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31453a5-3164-4f52-8ecd-ebb55153fc12 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:16.413479 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31453a5-3164-4f52-8ecd-ebb55153fc12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:16.423673 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31453a5-3164-4f52-8ecd-ebb55153fc12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:16.425212 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31453a5-3164-4f52-8ecd-ebb55153fc12 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:16.434648 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1293/2565] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:25:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:25:16.775031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e76c3-54cc-4f77-a1d9-0427b98c3914 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:16.776962 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-5e9e76c3-54cc-4f77-a1d9-0427b98c3914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:16.792291 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e76c3-54cc-4f77-a1d9-0427b98c3914 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:16.798194 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce82f9-e31f-4a67-a36f-c6dc40745074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:16.799148 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce82f9-e31f-4a67-a36f-c6dc40745074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:16.799496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce82f9-e31f-4a67-a36f-c6dc40745074 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:16.799976 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce82f9-e31f-4a67-a36f-c6dc40745074 admin admin] PATH_INFO: `/v3/projects/0e513bbf0edc4facaba307fa6116062d` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:16.801334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce82f9-e31f-4a67-a36f-c6dc40745074 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e513bbf0edc4facaba307fa6116062d)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:16.803368 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce82f9-e31f-4a67-a36f-c6dc40745074 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:16.854356 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-74ce82f9-e31f-4a67-a36f-c6dc40745074 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e513bbf0edc4facaba307fa6116062d', 'request_id': 'req-74ce82f9-e31f-4a67-a36f-c6dc40745074'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:16.860824 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-74ce82f9-e31f-4a67-a36f-c6dc40745074 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e513bbf0edc4facaba307fa6116062d', 'request_id': 'req-74ce82f9-e31f-4a67-a36f-c6dc40745074'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:16.866498 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-74ce82f9-e31f-4a67-a36f-c6dc40745074 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e513bbf0edc4facaba307fa6116062d', 'request_id': 'req-74ce82f9-e31f-4a67-a36f-c6dc40745074'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:16.880120 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-74ce82f9-e31f-4a67-a36f-c6dc40745074 admin admin] The token cache is being invalidate because project 0e513bbf0edc4facaba307fa6116062d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:25:16.880410 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-74ce82f9-e31f-4a67-a36f-c6dc40745074 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:16.881035 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-74ce82f9-e31f-4a67-a36f-c6dc40745074 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:16.882590 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1273/2566] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:25:16 2022] DELETE /identity/v3/projects/0e513bbf0edc4facaba307fa6116062d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:25:16.886790 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31453a5-3164-4f52-8ecd-ebb55153fc12 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:16.896237 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31453a5-3164-4f52-8ecd-ebb55153fc12 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:16.950243 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86f711-74ba-4a48-9406-2df2e8b25a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:16.951106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86f711-74ba-4a48-9406-2df2e8b25a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:16.951388 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86f711-74ba-4a48-9406-2df2e8b25a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:16.951659 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86f711-74ba-4a48-9406-2df2e8b25a6a admin admin] PATH_INFO: `/v3/users/919111f7a72f4e5d8e77bac3696aae48` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:16.952894 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f86f711-74ba-4a48-9406-2df2e8b25a6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=919111f7a72f4e5d8e77bac3696aae48)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:16.954826 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f86f711-74ba-4a48-9406-2df2e8b25a6a admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:17.007641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-5f86f711-74ba-4a48-9406-2df2e8b25a6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '919111f7a72f4e5d8e77bac3696aae48'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:17.015118 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-5f86f711-74ba-4a48-9406-2df2e8b25a6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '919111f7a72f4e5d8e77bac3696aae48'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:17.015821 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-5f86f711-74ba-4a48-9406-2df2e8b25a6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '919111f7a72f4e5d8e77bac3696aae48'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:17.023693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-5f86f711-74ba-4a48-9406-2df2e8b25a6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '919111f7a72f4e5d8e77bac3696aae48'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:17.024428 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-5f86f711-74ba-4a48-9406-2df2e8b25a6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '919111f7a72f4e5d8e77bac3696aae48'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:17.041987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1294/2567] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:25:16 2022] DELETE /identity/v3/users/919111f7a72f4e5d8e77bac3696aae48 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 10:25:17.334727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce82f9-e31f-4a67-a36f-c6dc40745074 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:17.336321 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-74ce82f9-e31f-4a67-a36f-c6dc40745074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:17.355921 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce82f9-e31f-4a67-a36f-c6dc40745074 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:17.411538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77e2f6-c943-4366-a9d3-14ed59e24bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:17.412395 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77e2f6-c943-4366-a9d3-14ed59e24bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:17.412690 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77e2f6-c943-4366-a9d3-14ed59e24bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:17.412967 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77e2f6-c943-4366-a9d3-14ed59e24bb6 admin admin] PATH_INFO: `/v3/projects/7df486e1b37a4d64ad5bd215c0c7f363` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:17.414282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77e2f6-c943-4366-a9d3-14ed59e24bb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7df486e1b37a4d64ad5bd215c0c7f363)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:17.416201 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77e2f6-c943-4366-a9d3-14ed59e24bb6 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:17.472354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6d77e2f6-c943-4366-a9d3-14ed59e24bb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7df486e1b37a4d64ad5bd215c0c7f363', 'request_id': 'req-6d77e2f6-c943-4366-a9d3-14ed59e24bb6'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:17.479114 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6d77e2f6-c943-4366-a9d3-14ed59e24bb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7df486e1b37a4d64ad5bd215c0c7f363', 'request_id': 'req-6d77e2f6-c943-4366-a9d3-14ed59e24bb6'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:17.485498 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6d77e2f6-c943-4366-a9d3-14ed59e24bb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7df486e1b37a4d64ad5bd215c0c7f363', 'request_id': 'req-6d77e2f6-c943-4366-a9d3-14ed59e24bb6'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:17.499182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6d77e2f6-c943-4366-a9d3-14ed59e24bb6 admin admin] The token cache is being invalidate because project 7df486e1b37a4d64ad5bd215c0c7f363 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:25:17.499446 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6d77e2f6-c943-4366-a9d3-14ed59e24bb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:17.500070 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6d77e2f6-c943-4366-a9d3-14ed59e24bb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:17.501723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1274/2568] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:25:17 2022] DELETE /identity/v3/projects/7df486e1b37a4d64ad5bd215c0c7f363 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:25:17.506770 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cc78f-6555-42ee-9228-9520db8d6c07 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:17.507276 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cc78f-6555-42ee-9228-9520db8d6c07 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:17.507468 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cc78f-6555-42ee-9228-9520db8d6c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:17.525510 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-939cc78f-6555-42ee-9228-9520db8d6c07 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:25:17.526712 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-939cc78f-6555-42ee-9228-9520db8d6c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:17.586992 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1295/2569] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:25:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:25:17.595014 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b61b07-bc6d-4532-ba0e-59254648027b None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:17.595298 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b61b07-bc6d-4532-ba0e-59254648027b None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:17.595563 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b61b07-bc6d-4532-ba0e-59254648027b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:17.614334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-82b61b07-bc6d-4532-ba0e-59254648027b None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:25:17.615558 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-82b61b07-bc6d-4532-ba0e-59254648027b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:17.623992 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1275/2570] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:25:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:25:17.628277 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939cc78f-6555-42ee-9228-9520db8d6c07 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:17.637814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939cc78f-6555-42ee-9228-9520db8d6c07 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:17.642517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c254c1e-9a31-408c-aab0-6a901478ffed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:17.643167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c254c1e-9a31-408c-aab0-6a901478ffed admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:17.643470 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c254c1e-9a31-408c-aab0-6a901478ffed admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:17.643744 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c254c1e-9a31-408c-aab0-6a901478ffed admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:17.644467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c254c1e-9a31-408c-aab0-6a901478ffed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 10:25:17.644798 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c254c1e-9a31-408c-aab0-6a901478ffed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:17.646420 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c254c1e-9a31-408c-aab0-6a901478ffed admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:17.657743 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1296/2571] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 10:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 10:25:17.662378 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b61b07-bc6d-4532-ba0e-59254648027b None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:17.671839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b61b07-bc6d-4532-ba0e-59254648027b None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:17.677000 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ad551-67d1-471f-81c0-c611b3799bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:17.677649 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ad551-67d1-471f-81c0-c611b3799bd4 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:17.677924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ad551-67d1-471f-81c0-c611b3799bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:17.678188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ad551-67d1-471f-81c0-c611b3799bd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:17.678904 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ad551-67d1-471f-81c0-c611b3799bd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:17.680838 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ad551-67d1-471f-81c0-c611b3799bd4 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:17.707932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1276/2572] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:25:17 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:25:17.712313 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c254c1e-9a31-408c-aab0-6a901478ffed admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:17.722398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c254c1e-9a31-408c-aab0-6a901478ffed admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:17.786614 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24e2f9-4e05-4646-a59a-2d89e6877d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:17.787358 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24e2f9-4e05-4646-a59a-2d89e6877d85 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:17.787641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24e2f9-4e05-4646-a59a-2d89e6877d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:17.787905 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24e2f9-4e05-4646-a59a-2d89e6877d85 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:17.788604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24e2f9-4e05-4646-a59a-2d89e6877d85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:17.790680 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24e2f9-4e05-4646-a59a-2d89e6877d85 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:17.811031 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1297/2573] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:25:17 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:25:17.815055 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ad551-67d1-471f-81c0-c611b3799bd4 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:17.825595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ad551-67d1-471f-81c0-c611b3799bd4 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:17.830338 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe1889-e489-466e-be40-0df03e1e2399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:17.831111 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe1889-e489-466e-be40-0df03e1e2399 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:17.831445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe1889-e489-466e-be40-0df03e1e2399 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:17.831776 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe1889-e489-466e-be40-0df03e1e2399 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:17.832563 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe1889-e489-466e-be40-0df03e1e2399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:17.834561 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe1889-e489-466e-be40-0df03e1e2399 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:17.845456 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1277/2574] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:25:17.849723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24e2f9-4e05-4646-a59a-2d89e6877d85 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:17.859991 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24e2f9-4e05-4646-a59a-2d89e6877d85 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:17.866129 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887556c-21a9-4504-ae25-d4266eaf776b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:17.866902 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887556c-21a9-4504-ae25-d4266eaf776b admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:17.867251 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887556c-21a9-4504-ae25-d4266eaf776b admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:17.867587 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887556c-21a9-4504-ae25-d4266eaf776b admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:17.868324 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887556c-21a9-4504-ae25-d4266eaf776b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:17.869990 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887556c-21a9-4504-ae25-d4266eaf776b admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:17.881400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1298/2575] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:25:17.886297 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe1889-e489-466e-be40-0df03e1e2399 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:17.897325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe1889-e489-466e-be40-0df03e1e2399 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:17.902346 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf1ec5-b868-4a7e-8d97-c229b0cd738a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:17.903237 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf1ec5-b868-4a7e-8d97-c229b0cd738a admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:17.903518 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf1ec5-b868-4a7e-8d97-c229b0cd738a admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:17.903791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf1ec5-b868-4a7e-8d97-c229b0cd738a admin admin] PATH_INFO: `/v3/projects/4ffe037928bf49bc9da491ff8d61bf28/users/f891898329b94fd19911a1f8b8f750a7/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:17.921058 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf1ec5-b868-4a7e-8d97-c229b0cd738a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ffe037928bf49bc9da491ff8d61bf28, user_id=f891898329b94fd19911a1f8b8f750a7, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:17.923722 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf1ec5-b868-4a7e-8d97-c229b0cd738a admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:17.942144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1278/2576] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:25:17 2022] PUT /identity/v3/projects/4ffe037928bf49bc9da491ff8d61bf28/users/f891898329b94fd19911a1f8b8f750a7/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:25:18.004280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a0180-42c7-4c70-8144-f7a4ba00379b None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:18.004762 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a0180-42c7-4c70-8144-f7a4ba00379b None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:18.005240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a0180-42c7-4c70-8144-f7a4ba00379b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:18.041386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-a43a0180-42c7-4c70-8144-f7a4ba00379b None None] MFA Rules not processed for user `f891898329b94fd19911a1f8b8f750a7`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:25:18.042628 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-a43a0180-42c7-4c70-8144-f7a4ba00379b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:18.114826 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1299/2577] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:25:18 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:25:18.119376 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf1ec5-b868-4a7e-8d97-c229b0cd738a admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:18.182299 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf1ec5-b868-4a7e-8d97-c229b0cd738a admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:18.189502 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b65da-f250-4f75-98ec-23da694d0c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:18.190501 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b65da-f250-4f75-98ec-23da694d0c99 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:18.190783 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b65da-f250-4f75-98ec-23da694d0c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:18.191056 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b65da-f250-4f75-98ec-23da694d0c99 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:18.192130 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b65da-f250-4f75-98ec-23da694d0c99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:18.194533 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b65da-f250-4f75-98ec-23da694d0c99 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:18.232448 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1279/2578] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:25:18 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:25:18.238373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a0180-42c7-4c70-8144-f7a4ba00379b None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:18.251272 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a0180-42c7-4c70-8144-f7a4ba00379b None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:18.318279 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a2b52-da4b-4235-b516-3209f3d37b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:18.318943 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a2b52-da4b-4235-b516-3209f3d37b4a admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:18.319218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a2b52-da4b-4235-b516-3209f3d37b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:18.319468 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a2b52-da4b-4235-b516-3209f3d37b4a admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:18.320154 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a2b52-da4b-4235-b516-3209f3d37b4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:18.322114 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a2b52-da4b-4235-b516-3209f3d37b4a admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:18.346745 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1300/2579] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:25:18 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 10:25:18.352626 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b65da-f250-4f75-98ec-23da694d0c99 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:18.366715 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b65da-f250-4f75-98ec-23da694d0c99 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:18.372167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c15afa4-70a2-4af9-bbae-71f0bf9539ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:18.372860 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15afa4-70a2-4af9-bbae-71f0bf9539ac admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:18.373136 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15afa4-70a2-4af9-bbae-71f0bf9539ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:18.373402 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15afa4-70a2-4af9-bbae-71f0bf9539ac admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:18.374123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c15afa4-70a2-4af9-bbae-71f0bf9539ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:18.375719 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c15afa4-70a2-4af9-bbae-71f0bf9539ac admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:18.386817 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1280/2580] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:25:18.391132 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a2b52-da4b-4235-b516-3209f3d37b4a admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:18.402073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a2b52-da4b-4235-b516-3209f3d37b4a admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:18.407392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9338a99-0eb8-46b7-b9f1-e41ea9a8e8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:18.408302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9338a99-0eb8-46b7-b9f1-e41ea9a8e8b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:18.408577 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9338a99-0eb8-46b7-b9f1-e41ea9a8e8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:18.408877 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9338a99-0eb8-46b7-b9f1-e41ea9a8e8b1 admin admin] PATH_INFO: `/v3/projects/42b02462bdd04ea1b4de12a1352a9460/users/885391d599184f57997db49edec2b512/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:18.423862 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9338a99-0eb8-46b7-b9f1-e41ea9a8e8b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42b02462bdd04ea1b4de12a1352a9460, user_id=885391d599184f57997db49edec2b512, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:18.426575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9338a99-0eb8-46b7-b9f1-e41ea9a8e8b1 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:18.443344 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1301/2581] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:25:18 2022] PUT /identity/v3/projects/42b02462bdd04ea1b4de12a1352a9460/users/885391d599184f57997db49edec2b512/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:25:18.449153 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c15afa4-70a2-4af9-bbae-71f0bf9539ac admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:18.464738 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c15afa4-70a2-4af9-bbae-71f0bf9539ac admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:18.529783 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd89c7c-c3ce-42f8-80b7-291053df59d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:18.530473 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd89c7c-c3ce-42f8-80b7-291053df59d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:18.530755 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd89c7c-c3ce-42f8-80b7-291053df59d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:18.531029 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd89c7c-c3ce-42f8-80b7-291053df59d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:18.531757 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd89c7c-c3ce-42f8-80b7-291053df59d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:18.533423 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd89c7c-c3ce-42f8-80b7-291053df59d8 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:18.547912 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1281/2582] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:25:18.552268 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9338a99-0eb8-46b7-b9f1-e41ea9a8e8b1 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:18.564040 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9338a99-0eb8-46b7-b9f1-e41ea9a8e8b1 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:18.570488 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298e704-1851-4194-80e5-599b4f2f180b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:18.571430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298e704-1851-4194-80e5-599b4f2f180b admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:18.571704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298e704-1851-4194-80e5-599b4f2f180b admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:18.571974 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298e704-1851-4194-80e5-599b4f2f180b admin admin] PATH_INFO: `/v3/projects/42b02462bdd04ea1b4de12a1352a9460/users/885391d599184f57997db49edec2b512/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:18.574330 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298e704-1851-4194-80e5-599b4f2f180b admin admin] RBAC: Authorizing `identity:create_grant(project_id=42b02462bdd04ea1b4de12a1352a9460, user_id=885391d599184f57997db49edec2b512, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:18.577147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298e704-1851-4194-80e5-599b4f2f180b admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:18.592162 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1302/2583] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:25:18 2022] PUT /identity/v3/projects/42b02462bdd04ea1b4de12a1352a9460/users/885391d599184f57997db49edec2b512/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:25:18.603044 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee09a1-7902-4d0f-8031-9c2eb7652e9b None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:18.603340 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee09a1-7902-4d0f-8031-9c2eb7652e9b None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:18.603621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee09a1-7902-4d0f-8031-9c2eb7652e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:18.641131 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-f2ee09a1-7902-4d0f-8031-9c2eb7652e9b None None] MFA Rules not processed for user `885391d599184f57997db49edec2b512`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:25:18.642307 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f2ee09a1-7902-4d0f-8031-9c2eb7652e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:18.715664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1282/2584] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:25:18 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:25:18.727003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298e704-1851-4194-80e5-599b4f2f180b admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:18.728781 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-2298e704-1851-4194-80e5-599b4f2f180b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:18.790542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298e704-1851-4194-80e5-599b4f2f180b admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:18.794673 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400ede7-6c56-4256-ae9d-69480cee7ca6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:18.795242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400ede7-6c56-4256-ae9d-69480cee7ca6 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:18.795519 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400ede7-6c56-4256-ae9d-69480cee7ca6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:18.795794 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400ede7-6c56-4256-ae9d-69480cee7ca6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:18.805678 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400ede7-6c56-4256-ae9d-69480cee7ca6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:18.807134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400ede7-6c56-4256-ae9d-69480cee7ca6 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:18.822751 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1303/2585] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:25:18 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:25:18.868898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee09a1-7902-4d0f-8031-9c2eb7652e9b None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:18.870343 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f2ee09a1-7902-4d0f-8031-9c2eb7652e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:18.883454 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee09a1-7902-4d0f-8031-9c2eb7652e9b None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:18.887911 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696452c8-c041-4d67-af4d-10034d080e6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:18.888525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696452c8-c041-4d67-af4d-10034d080e6a service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:18.888879 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696452c8-c041-4d67-af4d-10034d080e6a service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:18.889765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696452c8-c041-4d67-af4d-10034d080e6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:18.947985 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696452c8-c041-4d67-af4d-10034d080e6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:18.949457 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696452c8-c041-4d67-af4d-10034d080e6a service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:18.959832 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1283/2586] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:25:18 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:25:20.500907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de2f51-411c-48aa-9b66-57a060e0aef7 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:20.501569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de2f51-411c-48aa-9b66-57a060e0aef7 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:20.501700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de2f51-411c-48aa-9b66-57a060e0aef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:20.525382 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-14de2f51-411c-48aa-9b66-57a060e0aef7 None None] MFA Rules not processed for user `af62b2d3f73f482b8db4173af33752c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:25:20.526732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-14de2f51-411c-48aa-9b66-57a060e0aef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:20.591540 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1304/2587] 10.222.0.26 () {58 vars in 967 bytes} [Mon May 9 10:25:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:25:20.605411 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696452c8-c041-4d67-af4d-10034d080e6a service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:20.607045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-696452c8-c041-4d67-af4d-10034d080e6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:20.686266 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696452c8-c041-4d67-af4d-10034d080e6a service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:20.695681 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88959c9-9de9-4595-b857-fc29ba999b79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380e04358f2349aa912d6a97518877cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:20.696547 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88959c9-9de9-4595-b857-fc29ba999b79 service swift] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:20.696866 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88959c9-9de9-4595-b857-fc29ba999b79 service swift] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:20.697181 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88959c9-9de9-4595-b857-fc29ba999b79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:20.708653 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88959c9-9de9-4595-b857-fc29ba999b79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:20.710534 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88959c9-9de9-4595-b857-fc29ba999b79 service swift] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:20.720028 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1284/2588] 10.222.0.26 () {60 vars in 1384 bytes} [Mon May 9 10:25:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 10:25:22.309256 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de2f51-411c-48aa-9b66-57a060e0aef7 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:22.375198 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de2f51-411c-48aa-9b66-57a060e0aef7 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:22.380332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e271441-5344-49d5-a731-72c3cf7cb782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:22.381246 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e271441-5344-49d5-a731-72c3cf7cb782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:22.381613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e271441-5344-49d5-a731-72c3cf7cb782 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:22.381781 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e271441-5344-49d5-a731-72c3cf7cb782 admin admin] PATH_INFO: `/v3/users/f891898329b94fd19911a1f8b8f750a7` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:22.383037 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e271441-5344-49d5-a731-72c3cf7cb782 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f891898329b94fd19911a1f8b8f750a7)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:22.385007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e271441-5344-49d5-a731-72c3cf7cb782 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:22.438095 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-6e271441-5344-49d5-a731-72c3cf7cb782 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f891898329b94fd19911a1f8b8f750a7'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:22.444448 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-6e271441-5344-49d5-a731-72c3cf7cb782 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f891898329b94fd19911a1f8b8f750a7'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:22.445137 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-6e271441-5344-49d5-a731-72c3cf7cb782 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f891898329b94fd19911a1f8b8f750a7'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:22.453070 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-6e271441-5344-49d5-a731-72c3cf7cb782 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f891898329b94fd19911a1f8b8f750a7'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:22.453664 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-6e271441-5344-49d5-a731-72c3cf7cb782 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f891898329b94fd19911a1f8b8f750a7'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:22.470015 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1305/2589] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:25:22 2022] DELETE /identity/v3/users/f891898329b94fd19911a1f8b8f750a7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:25:22.474776 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a7c30-6195-49fb-a6d4-1ff35224143c None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:22.475258 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a7c30-6195-49fb-a6d4-1ff35224143c None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:22.475379 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a7c30-6195-49fb-a6d4-1ff35224143c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:22.493174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-d23a7c30-6195-49fb-a6d4-1ff35224143c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:25:22.494391 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-d23a7c30-6195-49fb-a6d4-1ff35224143c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:22.559038 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1285/2590] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 10:25:22.569030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e271441-5344-49d5-a731-72c3cf7cb782 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:22.570522 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-6e271441-5344-49d5-a731-72c3cf7cb782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:22.637945 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e271441-5344-49d5-a731-72c3cf7cb782 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:22.641982 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18551256-cc70-4a69-8117-1146295c0d9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:22.643252 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18551256-cc70-4a69-8117-1146295c0d9f service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:22.643568 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18551256-cc70-4a69-8117-1146295c0d9f service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:22.643833 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18551256-cc70-4a69-8117-1146295c0d9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:22.653073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18551256-cc70-4a69-8117-1146295c0d9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:22.654497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18551256-cc70-4a69-8117-1146295c0d9f service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:22.664007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1306/2591] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:25:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:25:22.951070 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a7c30-6195-49fb-a6d4-1ff35224143c None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:22.952599 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-d23a7c30-6195-49fb-a6d4-1ff35224143c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:22.965236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a7c30-6195-49fb-a6d4-1ff35224143c None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:22.970531 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de6f8f-7135-47c5-af43-d8c822627001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:22.971382 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de6f8f-7135-47c5-af43-d8c822627001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:22.971747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de6f8f-7135-47c5-af43-d8c822627001 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:22.972018 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de6f8f-7135-47c5-af43-d8c822627001 admin admin] PATH_INFO: `/v3/projects/4ffe037928bf49bc9da491ff8d61bf28` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:22.973245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de6f8f-7135-47c5-af43-d8c822627001 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ffe037928bf49bc9da491ff8d61bf28)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:22.975162 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de6f8f-7135-47c5-af43-d8c822627001 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:23.025250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-35de6f8f-7135-47c5-af43-d8c822627001 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ffe037928bf49bc9da491ff8d61bf28', 'request_id': 'req-35de6f8f-7135-47c5-af43-d8c822627001'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:23.031516 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-35de6f8f-7135-47c5-af43-d8c822627001 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ffe037928bf49bc9da491ff8d61bf28', 'request_id': 'req-35de6f8f-7135-47c5-af43-d8c822627001'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:23.037395 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-35de6f8f-7135-47c5-af43-d8c822627001 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ffe037928bf49bc9da491ff8d61bf28', 'request_id': 'req-35de6f8f-7135-47c5-af43-d8c822627001'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:23.052686 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-35de6f8f-7135-47c5-af43-d8c822627001 admin admin] The token cache is being invalidate because project 4ffe037928bf49bc9da491ff8d61bf28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:25:23.052941 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-35de6f8f-7135-47c5-af43-d8c822627001 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:23.053612 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-35de6f8f-7135-47c5-af43-d8c822627001 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:23.055183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1286/2592] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:25:22 2022] DELETE /identity/v3/projects/4ffe037928bf49bc9da491ff8d61bf28 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:25:23.059221 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18551256-cc70-4a69-8117-1146295c0d9f service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:23.069086 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18551256-cc70-4a69-8117-1146295c0d9f service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:23.135045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68846ca2-f298-4032-8d22-985aa8f1a18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:23.135915 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68846ca2-f298-4032-8d22-985aa8f1a18d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:23.136194 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68846ca2-f298-4032-8d22-985aa8f1a18d admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:23.136462 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68846ca2-f298-4032-8d22-985aa8f1a18d admin admin] PATH_INFO: `/v3/users/885391d599184f57997db49edec2b512` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:23.137752 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68846ca2-f298-4032-8d22-985aa8f1a18d admin admin] RBAC: Authorizing `identity:delete_user(user_id=885391d599184f57997db49edec2b512)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:23.139591 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68846ca2-f298-4032-8d22-985aa8f1a18d admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:23.192052 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-68846ca2-f298-4032-8d22-985aa8f1a18d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '885391d599184f57997db49edec2b512'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:23.198585 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-68846ca2-f298-4032-8d22-985aa8f1a18d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '885391d599184f57997db49edec2b512'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:23.199241 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-68846ca2-f298-4032-8d22-985aa8f1a18d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '885391d599184f57997db49edec2b512'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:23.207625 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-68846ca2-f298-4032-8d22-985aa8f1a18d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '885391d599184f57997db49edec2b512'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:23.208759 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-68846ca2-f298-4032-8d22-985aa8f1a18d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '885391d599184f57997db49edec2b512'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:23.226466 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1307/2593] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:25:23 2022] DELETE /identity/v3/users/885391d599184f57997db49edec2b512 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:25:23.500013 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de6f8f-7135-47c5-af43-d8c822627001 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:23.501527 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-35de6f8f-7135-47c5-af43-d8c822627001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:23.514698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de6f8f-7135-47c5-af43-d8c822627001 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:23.573218 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735eff04-21a7-47b5-a560-4df3d2db552d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:23.574072 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735eff04-21a7-47b5-a560-4df3d2db552d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:23.574345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735eff04-21a7-47b5-a560-4df3d2db552d admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:23.574613 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735eff04-21a7-47b5-a560-4df3d2db552d admin admin] PATH_INFO: `/v3/projects/42b02462bdd04ea1b4de12a1352a9460` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:23.575911 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735eff04-21a7-47b5-a560-4df3d2db552d admin admin] RBAC: Authorizing `identity:delete_project(project_id=42b02462bdd04ea1b4de12a1352a9460)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:23.577838 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735eff04-21a7-47b5-a560-4df3d2db552d admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:23.632446 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-735eff04-21a7-47b5-a560-4df3d2db552d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42b02462bdd04ea1b4de12a1352a9460', 'request_id': 'req-735eff04-21a7-47b5-a560-4df3d2db552d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:23.638343 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-735eff04-21a7-47b5-a560-4df3d2db552d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42b02462bdd04ea1b4de12a1352a9460', 'request_id': 'req-735eff04-21a7-47b5-a560-4df3d2db552d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:23.646251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-735eff04-21a7-47b5-a560-4df3d2db552d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42b02462bdd04ea1b4de12a1352a9460', 'request_id': 'req-735eff04-21a7-47b5-a560-4df3d2db552d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:23.661605 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-735eff04-21a7-47b5-a560-4df3d2db552d admin admin] The token cache is being invalidate because project 42b02462bdd04ea1b4de12a1352a9460 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:25:23.661885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-735eff04-21a7-47b5-a560-4df3d2db552d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:23.662483 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-735eff04-21a7-47b5-a560-4df3d2db552d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:23.664039 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1287/2594] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:25:23 2022] DELETE /identity/v3/projects/42b02462bdd04ea1b4de12a1352a9460 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:25:23.669098 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04f034-7961-4b0e-911a-c8d96356be5e None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:23.669618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04f034-7961-4b0e-911a-c8d96356be5e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:23.669618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04f034-7961-4b0e-911a-c8d96356be5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:23.689240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-ba04f034-7961-4b0e-911a-c8d96356be5e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:25:23.690874 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-ba04f034-7961-4b0e-911a-c8d96356be5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:23.752147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1308/2595] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:25:23.760289 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd8f2b-7bae-4bb3-b158-637cf928a8c6 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:23.760570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd8f2b-7bae-4bb3-b158-637cf928a8c6 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:23.760872 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd8f2b-7bae-4bb3-b158-637cf928a8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:23.780555 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-8fdd8f2b-7bae-4bb3-b158-637cf928a8c6 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:25:23.781803 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-8fdd8f2b-7bae-4bb3-b158-637cf928a8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:23.790235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1288/2596] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:25:23.795241 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04f034-7961-4b0e-911a-c8d96356be5e None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:23.810051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04f034-7961-4b0e-911a-c8d96356be5e None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:23.817261 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68350ec7-7443-4914-a1b7-32b31e096ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:23.817913 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68350ec7-7443-4914-a1b7-32b31e096ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:23.818198 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68350ec7-7443-4914-a1b7-32b31e096ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:23.818526 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68350ec7-7443-4914-a1b7-32b31e096ee1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:23.819478 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68350ec7-7443-4914-a1b7-32b31e096ee1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 10:25:23.819804 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68350ec7-7443-4914-a1b7-32b31e096ee1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:23.821932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68350ec7-7443-4914-a1b7-32b31e096ee1 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:23.837275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1309/2597] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 10:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 10:25:23.842058 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd8f2b-7bae-4bb3-b158-637cf928a8c6 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:23.852244 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd8f2b-7bae-4bb3-b158-637cf928a8c6 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:23.857216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73280c4a-9204-46e3-8001-ba85bfc1207e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:23.857876 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73280c4a-9204-46e3-8001-ba85bfc1207e admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:23.858206 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73280c4a-9204-46e3-8001-ba85bfc1207e admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:23.858484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73280c4a-9204-46e3-8001-ba85bfc1207e admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:23.859177 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73280c4a-9204-46e3-8001-ba85bfc1207e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:23.861116 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73280c4a-9204-46e3-8001-ba85bfc1207e admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:23.888258 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1289/2598] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:25:23 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:25:23.893094 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68350ec7-7443-4914-a1b7-32b31e096ee1 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:23.907783 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68350ec7-7443-4914-a1b7-32b31e096ee1 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:23.976433 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005e403-7e8e-418f-844f-a3d55e764581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:23.977125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005e403-7e8e-418f-844f-a3d55e764581 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:23.977401 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005e403-7e8e-418f-844f-a3d55e764581 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:23.977691 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005e403-7e8e-418f-844f-a3d55e764581 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:23.978364 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005e403-7e8e-418f-844f-a3d55e764581 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:23.980339 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005e403-7e8e-418f-844f-a3d55e764581 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:24.000337 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1310/2599] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:25:23 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:25:24.004580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73280c4a-9204-46e3-8001-ba85bfc1207e admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:24.014794 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73280c4a-9204-46e3-8001-ba85bfc1207e admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:24.019771 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc739099-26ca-49dc-a501-a4ab9c41bbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:24.020428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc739099-26ca-49dc-a501-a4ab9c41bbe3 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:24.020731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc739099-26ca-49dc-a501-a4ab9c41bbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:24.021026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc739099-26ca-49dc-a501-a4ab9c41bbe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:24.021775 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc739099-26ca-49dc-a501-a4ab9c41bbe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:24.023383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc739099-26ca-49dc-a501-a4ab9c41bbe3 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:24.033706 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1290/2600] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:25:24.038096 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005e403-7e8e-418f-844f-a3d55e764581 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:24.048825 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005e403-7e8e-418f-844f-a3d55e764581 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:24.054278 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c08c3-2166-483f-afd7-00d2c639078a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:24.054937 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c08c3-2166-483f-afd7-00d2c639078a admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:24.055211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c08c3-2166-483f-afd7-00d2c639078a admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:24.055477 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c08c3-2166-483f-afd7-00d2c639078a admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:24.056198 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c08c3-2166-483f-afd7-00d2c639078a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:24.057825 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c08c3-2166-483f-afd7-00d2c639078a admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:24.068878 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1311/2601] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:25:24.074770 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc739099-26ca-49dc-a501-a4ab9c41bbe3 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:24.085595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc739099-26ca-49dc-a501-a4ab9c41bbe3 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:24.091504 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea5cc3-bde7-4480-a83b-ef0c6adcb9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:24.092383 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea5cc3-bde7-4480-a83b-ef0c6adcb9cb admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:24.092651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea5cc3-bde7-4480-a83b-ef0c6adcb9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:24.092951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea5cc3-bde7-4480-a83b-ef0c6adcb9cb admin admin] PATH_INFO: `/v3/projects/9a71a619d8aa4f24bd2e827512b56e7a/users/8298b742cbe042fc911aa09555d6c21c/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:24.109202 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea5cc3-bde7-4480-a83b-ef0c6adcb9cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a71a619d8aa4f24bd2e827512b56e7a, user_id=8298b742cbe042fc911aa09555d6c21c, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:24.111893 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea5cc3-bde7-4480-a83b-ef0c6adcb9cb admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:24.126962 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1291/2602] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:25:24 2022] PUT /identity/v3/projects/9a71a619d8aa4f24bd2e827512b56e7a/users/8298b742cbe042fc911aa09555d6c21c/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:25:24.138182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738335d-d86b-49bd-8719-e1c363cbdd6f None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:24.138464 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738335d-d86b-49bd-8719-e1c363cbdd6f None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:24.138732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738335d-d86b-49bd-8719-e1c363cbdd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:24.165279 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-1738335d-d86b-49bd-8719-e1c363cbdd6f None None] MFA Rules not processed for user `8298b742cbe042fc911aa09555d6c21c`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:25:24.166485 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-1738335d-d86b-49bd-8719-e1c363cbdd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:24.247101 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1312/2603] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:25:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:25:24.251920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea5cc3-bde7-4480-a83b-ef0c6adcb9cb admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:24.322074 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea5cc3-bde7-4480-a83b-ef0c6adcb9cb admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:24.330909 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1744430-42b1-4239-b3ec-cdf0718fd10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:24.331563 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1744430-42b1-4239-b3ec-cdf0718fd10c admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:24.331834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1744430-42b1-4239-b3ec-cdf0718fd10c admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:24.332097 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1744430-42b1-4239-b3ec-cdf0718fd10c admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:24.332815 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1744430-42b1-4239-b3ec-cdf0718fd10c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:24.335063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1744430-42b1-4239-b3ec-cdf0718fd10c admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:24.362177 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1292/2604] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:25:24 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:25:24.366549 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738335d-d86b-49bd-8719-e1c363cbdd6f None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:24.377667 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738335d-d86b-49bd-8719-e1c363cbdd6f None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:24.439806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185227a-bb5f-4abf-aaef-a8049c1ca322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:24.440495 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185227a-bb5f-4abf-aaef-a8049c1ca322 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:24.440798 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185227a-bb5f-4abf-aaef-a8049c1ca322 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:24.441064 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185227a-bb5f-4abf-aaef-a8049c1ca322 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:24.441753 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3185227a-bb5f-4abf-aaef-a8049c1ca322 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:24.443640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3185227a-bb5f-4abf-aaef-a8049c1ca322 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:24.465016 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1313/2605] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:25:24 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:25:24.469164 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1744430-42b1-4239-b3ec-cdf0718fd10c admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:24.484256 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1744430-42b1-4239-b3ec-cdf0718fd10c admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:24.489406 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b976cb7-d88c-4fdf-9665-db2d35af4ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:24.490141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b976cb7-d88c-4fdf-9665-db2d35af4ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:24.490418 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b976cb7-d88c-4fdf-9665-db2d35af4ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:24.490688 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b976cb7-d88c-4fdf-9665-db2d35af4ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:24.491416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b976cb7-d88c-4fdf-9665-db2d35af4ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:24.493066 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b976cb7-d88c-4fdf-9665-db2d35af4ba2 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:24.504026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1293/2606] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:25:24.508728 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185227a-bb5f-4abf-aaef-a8049c1ca322 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:24.524330 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185227a-bb5f-4abf-aaef-a8049c1ca322 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:24.533355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef2b2d-623f-4333-ac05-de9bc570c4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:24.534443 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef2b2d-623f-4333-ac05-de9bc570c4f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:24.534719 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef2b2d-623f-4333-ac05-de9bc570c4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:24.534982 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef2b2d-623f-4333-ac05-de9bc570c4f1 admin admin] PATH_INFO: `/v3/projects/b788ae39702f4762ae925198f6bfa867/users/821915b2aa0943b6a9965ebfd9333659/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:24.551675 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef2b2d-623f-4333-ac05-de9bc570c4f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b788ae39702f4762ae925198f6bfa867, user_id=821915b2aa0943b6a9965ebfd9333659, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:24.554407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef2b2d-623f-4333-ac05-de9bc570c4f1 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:24.570101 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1314/2607] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:25:24 2022] PUT /identity/v3/projects/b788ae39702f4762ae925198f6bfa867/users/821915b2aa0943b6a9965ebfd9333659/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:25:24.574692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b976cb7-d88c-4fdf-9665-db2d35af4ba2 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:24.585361 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b976cb7-d88c-4fdf-9665-db2d35af4ba2 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:24.645893 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bf983-9838-451a-bc41-3f2297066f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:24.646554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bf983-9838-451a-bc41-3f2297066f3c admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:24.646813 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bf983-9838-451a-bc41-3f2297066f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:24.647116 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bf983-9838-451a-bc41-3f2297066f3c admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:24.647833 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76bf983-9838-451a-bc41-3f2297066f3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:24.649484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76bf983-9838-451a-bc41-3f2297066f3c admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:24.660202 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1294/2608] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:25:24.664708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef2b2d-623f-4333-ac05-de9bc570c4f1 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:24.676311 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef2b2d-623f-4333-ac05-de9bc570c4f1 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:24.681623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44203e9-63f3-4187-add7-8a5d0dca9f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:24.682537 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44203e9-63f3-4187-add7-8a5d0dca9f2b admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:24.682813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44203e9-63f3-4187-add7-8a5d0dca9f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:24.683084 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44203e9-63f3-4187-add7-8a5d0dca9f2b admin admin] PATH_INFO: `/v3/projects/b788ae39702f4762ae925198f6bfa867/users/821915b2aa0943b6a9965ebfd9333659/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:24.685231 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44203e9-63f3-4187-add7-8a5d0dca9f2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b788ae39702f4762ae925198f6bfa867, user_id=821915b2aa0943b6a9965ebfd9333659, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:24.687894 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44203e9-63f3-4187-add7-8a5d0dca9f2b admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:24.698641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1315/2609] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:25:24 2022] PUT /identity/v3/projects/b788ae39702f4762ae925198f6bfa867/users/821915b2aa0943b6a9965ebfd9333659/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:25:24.709205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af90cfe-d5c1-4cf9-9abf-2b8134961e20 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:24.709493 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af90cfe-d5c1-4cf9-9abf-2b8134961e20 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:24.709761 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af90cfe-d5c1-4cf9-9abf-2b8134961e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:24.736291 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-8af90cfe-d5c1-4cf9-9abf-2b8134961e20 None None] MFA Rules not processed for user `821915b2aa0943b6a9965ebfd9333659`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:25:24.737600 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-8af90cfe-d5c1-4cf9-9abf-2b8134961e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:24.821444 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1295/2610] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:25:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:25:24.834265 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44203e9-63f3-4187-add7-8a5d0dca9f2b admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:24.836250 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c44203e9-63f3-4187-add7-8a5d0dca9f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:24.895364 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44203e9-63f3-4187-add7-8a5d0dca9f2b admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:24.902542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92a0cb-b105-47dc-8b51-cccde3337e22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:24.903492 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92a0cb-b105-47dc-8b51-cccde3337e22 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:24.903920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92a0cb-b105-47dc-8b51-cccde3337e22 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:24.904353 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92a0cb-b105-47dc-8b51-cccde3337e22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:24.971163 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92a0cb-b105-47dc-8b51-cccde3337e22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:24.972618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92a0cb-b105-47dc-8b51-cccde3337e22 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:24.982033 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1316/2611] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:25:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:25:29.234860 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af90cfe-d5c1-4cf9-9abf-2b8134961e20 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:29.246751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af90cfe-d5c1-4cf9-9abf-2b8134961e20 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:29.251848 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e62ef23-9aff-4e06-b352-9d38380abf5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:29.252695 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62ef23-9aff-4e06-b352-9d38380abf5c service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:29.252961 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62ef23-9aff-4e06-b352-9d38380abf5c service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:29.253247 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62ef23-9aff-4e06-b352-9d38380abf5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:29.263674 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e62ef23-9aff-4e06-b352-9d38380abf5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:29.265145 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e62ef23-9aff-4e06-b352-9d38380abf5c service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:29.274993 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1296/2612] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:25:29 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:25:34.199070 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92a0cb-b105-47dc-8b51-cccde3337e22 service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:34.269430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92a0cb-b105-47dc-8b51-cccde3337e22 service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:34.275092 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaeec12-408a-43aa-b38d-b05e0c1a6e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:34.275955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaeec12-408a-43aa-b38d-b05e0c1a6e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:34.276207 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaeec12-408a-43aa-b38d-b05e0c1a6e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:34.276476 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaeec12-408a-43aa-b38d-b05e0c1a6e23 admin admin] PATH_INFO: `/v3/users/8298b742cbe042fc911aa09555d6c21c` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:34.277823 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaeec12-408a-43aa-b38d-b05e0c1a6e23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8298b742cbe042fc911aa09555d6c21c)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:34.279727 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaeec12-408a-43aa-b38d-b05e0c1a6e23 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:34.327829 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bcaeec12-408a-43aa-b38d-b05e0c1a6e23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8298b742cbe042fc911aa09555d6c21c'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:34.334798 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bcaeec12-408a-43aa-b38d-b05e0c1a6e23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8298b742cbe042fc911aa09555d6c21c'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:34.335462 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bcaeec12-408a-43aa-b38d-b05e0c1a6e23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8298b742cbe042fc911aa09555d6c21c'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:34.343966 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bcaeec12-408a-43aa-b38d-b05e0c1a6e23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8298b742cbe042fc911aa09555d6c21c'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:34.344575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-bcaeec12-408a-43aa-b38d-b05e0c1a6e23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8298b742cbe042fc911aa09555d6c21c'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:34.361643 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1317/2613] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:25:34 2022] DELETE /identity/v3/users/8298b742cbe042fc911aa09555d6c21c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:25:34.366281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78f6d3-d18e-41db-adc5-9bc0cda2610c None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:34.366818 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78f6d3-d18e-41db-adc5-9bc0cda2610c None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:34.366818 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78f6d3-d18e-41db-adc5-9bc0cda2610c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:34.389689 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-1c78f6d3-d18e-41db-adc5-9bc0cda2610c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:25:34.390925 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-1c78f6d3-d18e-41db-adc5-9bc0cda2610c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:34.455590 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1297/2614] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:25:34.466813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaeec12-408a-43aa-b38d-b05e0c1a6e23 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:34.468343 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-bcaeec12-408a-43aa-b38d-b05e0c1a6e23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:34.525661 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaeec12-408a-43aa-b38d-b05e0c1a6e23 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:34.529908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e95165-03b4-47c1-8d31-128f06ba0bb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:34.530539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e95165-03b4-47c1-8d31-128f06ba0bb5 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:34.530814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e95165-03b4-47c1-8d31-128f06ba0bb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:34.531075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e95165-03b4-47c1-8d31-128f06ba0bb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:34.541789 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e95165-03b4-47c1-8d31-128f06ba0bb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:34.543365 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e95165-03b4-47c1-8d31-128f06ba0bb5 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:34.555236 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1318/2615] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:25:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:25:34.839405 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78f6d3-d18e-41db-adc5-9bc0cda2610c None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:34.841050 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-1c78f6d3-d18e-41db-adc5-9bc0cda2610c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:34.852910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78f6d3-d18e-41db-adc5-9bc0cda2610c None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:34.857454 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fff43e-01dd-47e5-a712-e6d33c6acf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:34.858310 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fff43e-01dd-47e5-a712-e6d33c6acf77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:34.858585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fff43e-01dd-47e5-a712-e6d33c6acf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:34.858860 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fff43e-01dd-47e5-a712-e6d33c6acf77 admin admin] PATH_INFO: `/v3/projects/9a71a619d8aa4f24bd2e827512b56e7a` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:34.860075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fff43e-01dd-47e5-a712-e6d33c6acf77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a71a619d8aa4f24bd2e827512b56e7a)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:34.862194 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fff43e-01dd-47e5-a712-e6d33c6acf77 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:34.912693 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-72fff43e-01dd-47e5-a712-e6d33c6acf77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a71a619d8aa4f24bd2e827512b56e7a', 'request_id': 'req-72fff43e-01dd-47e5-a712-e6d33c6acf77'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:34.919264 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-72fff43e-01dd-47e5-a712-e6d33c6acf77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a71a619d8aa4f24bd2e827512b56e7a', 'request_id': 'req-72fff43e-01dd-47e5-a712-e6d33c6acf77'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:34.925085 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-72fff43e-01dd-47e5-a712-e6d33c6acf77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a71a619d8aa4f24bd2e827512b56e7a', 'request_id': 'req-72fff43e-01dd-47e5-a712-e6d33c6acf77'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:34.940320 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-72fff43e-01dd-47e5-a712-e6d33c6acf77 admin admin] The token cache is being invalidate because project 9a71a619d8aa4f24bd2e827512b56e7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:25:34.940609 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-72fff43e-01dd-47e5-a712-e6d33c6acf77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:34.941365 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-72fff43e-01dd-47e5-a712-e6d33c6acf77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:34.943101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1298/2616] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:25:34 2022] DELETE /identity/v3/projects/9a71a619d8aa4f24bd2e827512b56e7a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:25:34.947182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e95165-03b4-47c1-8d31-128f06ba0bb5 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:34.957818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e95165-03b4-47c1-8d31-128f06ba0bb5 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:35.014688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3c6b3-700e-4350-b631-c93ff1020a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:35.015754 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3c6b3-700e-4350-b631-c93ff1020a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:35.016083 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3c6b3-700e-4350-b631-c93ff1020a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:35.016396 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3c6b3-700e-4350-b631-c93ff1020a0b admin admin] PATH_INFO: `/v3/users/821915b2aa0943b6a9965ebfd9333659` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:35.018185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3c6b3-700e-4350-b631-c93ff1020a0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=821915b2aa0943b6a9965ebfd9333659)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:35.020174 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3c6b3-700e-4350-b631-c93ff1020a0b admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:35.070775 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-2cf3c6b3-700e-4350-b631-c93ff1020a0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '821915b2aa0943b6a9965ebfd9333659'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:35.077370 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-2cf3c6b3-700e-4350-b631-c93ff1020a0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '821915b2aa0943b6a9965ebfd9333659'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:35.078016 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-2cf3c6b3-700e-4350-b631-c93ff1020a0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '821915b2aa0943b6a9965ebfd9333659'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:35.086519 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-2cf3c6b3-700e-4350-b631-c93ff1020a0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '821915b2aa0943b6a9965ebfd9333659'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:35.087247 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-2cf3c6b3-700e-4350-b631-c93ff1020a0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '821915b2aa0943b6a9965ebfd9333659'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:35.104283 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1319/2617] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:25:34 2022] DELETE /identity/v3/users/821915b2aa0943b6a9965ebfd9333659 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:25:35.396643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fff43e-01dd-47e5-a712-e6d33c6acf77 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:35.398344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-72fff43e-01dd-47e5-a712-e6d33c6acf77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:35.412090 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fff43e-01dd-47e5-a712-e6d33c6acf77 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:35.467871 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbac9e-1142-452f-bc4a-43933ff67ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:35.468640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbac9e-1142-452f-bc4a-43933ff67ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:35.468953 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbac9e-1142-452f-bc4a-43933ff67ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:35.469217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbac9e-1142-452f-bc4a-43933ff67ad9 admin admin] PATH_INFO: `/v3/projects/b788ae39702f4762ae925198f6bfa867` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:35.470409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cbac9e-1142-452f-bc4a-43933ff67ad9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b788ae39702f4762ae925198f6bfa867)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:35.472339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cbac9e-1142-452f-bc4a-43933ff67ad9 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:35.519268 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-88cbac9e-1142-452f-bc4a-43933ff67ad9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b788ae39702f4762ae925198f6bfa867', 'request_id': 'req-88cbac9e-1142-452f-bc4a-43933ff67ad9'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:35.525800 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-88cbac9e-1142-452f-bc4a-43933ff67ad9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b788ae39702f4762ae925198f6bfa867', 'request_id': 'req-88cbac9e-1142-452f-bc4a-43933ff67ad9'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:35.532110 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-88cbac9e-1142-452f-bc4a-43933ff67ad9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b788ae39702f4762ae925198f6bfa867', 'request_id': 'req-88cbac9e-1142-452f-bc4a-43933ff67ad9'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:35.546809 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-88cbac9e-1142-452f-bc4a-43933ff67ad9 admin admin] The token cache is being invalidate because project b788ae39702f4762ae925198f6bfa867 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:25:35.547092 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-88cbac9e-1142-452f-bc4a-43933ff67ad9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:35.547772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-88cbac9e-1142-452f-bc4a-43933ff67ad9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:25:35.549466 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1299/2618] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:25:35 2022] DELETE /identity/v3/projects/b788ae39702f4762ae925198f6bfa867 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:25:35.555536 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe833d54-1b41-4ee3-b481-da21beef95ab None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:35.556055 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe833d54-1b41-4ee3-b481-da21beef95ab None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:35.556055 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe833d54-1b41-4ee3-b481-da21beef95ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:35.575056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-fe833d54-1b41-4ee3-b481-da21beef95ab None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:25:35.576302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-fe833d54-1b41-4ee3-b481-da21beef95ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:35.642108 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1320/2619] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:25:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:25:35.655212 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce9aa1-b13f-4a6a-9df0-8a13b65447ab None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:35.655522 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce9aa1-b13f-4a6a-9df0-8a13b65447ab None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:35.655875 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce9aa1-b13f-4a6a-9df0-8a13b65447ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:35.684003 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-38ce9aa1-b13f-4a6a-9df0-8a13b65447ab None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:25:35.685757 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-38ce9aa1-b13f-4a6a-9df0-8a13b65447ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:35.695568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1300/2620] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:25:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:25:35.700268 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe833d54-1b41-4ee3-b481-da21beef95ab None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:35.711962 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe833d54-1b41-4ee3-b481-da21beef95ab None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:35.718655 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd3e1b-c57d-4987-b382-1a42ced2f22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:35.719249 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd3e1b-c57d-4987-b382-1a42ced2f22d admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:35.719522 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd3e1b-c57d-4987-b382-1a42ced2f22d admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:35.720000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd3e1b-c57d-4987-b382-1a42ced2f22d admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:35.721420 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd3e1b-c57d-4987-b382-1a42ced2f22d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 10:25:35.721759 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd3e1b-c57d-4987-b382-1a42ced2f22d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:35.723376 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd3e1b-c57d-4987-b382-1a42ced2f22d admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:35.739325 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1321/2621] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 10:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 10:25:35.745975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce9aa1-b13f-4a6a-9df0-8a13b65447ab None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:35.758319 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce9aa1-b13f-4a6a-9df0-8a13b65447ab None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:35.766221 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6820fe-de97-4fae-835c-459e61fcd29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:35.767245 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6820fe-de97-4fae-835c-459e61fcd29d admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:35.767689 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6820fe-de97-4fae-835c-459e61fcd29d admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:35.768084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6820fe-de97-4fae-835c-459e61fcd29d admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:35.769151 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6820fe-de97-4fae-835c-459e61fcd29d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:35.771740 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6820fe-de97-4fae-835c-459e61fcd29d admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:35.804972 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1301/2622] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:25:35 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:25:35.809391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd3e1b-c57d-4987-b382-1a42ced2f22d admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:35.819343 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd3e1b-c57d-4987-b382-1a42ced2f22d admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:35.875720 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933026c5-d724-4e32-8148-fbee7949e58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:35.876375 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933026c5-d724-4e32-8148-fbee7949e58b admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:35.876689 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933026c5-d724-4e32-8148-fbee7949e58b admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:35.876956 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933026c5-d724-4e32-8148-fbee7949e58b admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:35.877678 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933026c5-d724-4e32-8148-fbee7949e58b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:35.879576 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933026c5-d724-4e32-8148-fbee7949e58b admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:35.907197 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1322/2623] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:25:35 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:25:35.912859 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6820fe-de97-4fae-835c-459e61fcd29d admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:35.923144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6820fe-de97-4fae-835c-459e61fcd29d admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:35.928079 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb63399-2475-4767-a714-9267876ccbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:35.928759 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb63399-2475-4767-a714-9267876ccbd3 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:35.929037 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb63399-2475-4767-a714-9267876ccbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:35.929306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb63399-2475-4767-a714-9267876ccbd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:35.930057 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb63399-2475-4767-a714-9267876ccbd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:35.931655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb63399-2475-4767-a714-9267876ccbd3 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:35.942102 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1302/2624] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:25:35.946898 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933026c5-d724-4e32-8148-fbee7949e58b admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:35.957187 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933026c5-d724-4e32-8148-fbee7949e58b admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:35.962335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98961f5f-c395-4878-9584-b9574185219b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:35.962994 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98961f5f-c395-4878-9584-b9574185219b admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:35.963267 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98961f5f-c395-4878-9584-b9574185219b admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:35.963551 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98961f5f-c395-4878-9584-b9574185219b admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:35.964299 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98961f5f-c395-4878-9584-b9574185219b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:35.965942 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98961f5f-c395-4878-9584-b9574185219b admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:35.976986 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1323/2625] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:25:35.982878 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb63399-2475-4767-a714-9267876ccbd3 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:35.994622 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb63399-2475-4767-a714-9267876ccbd3 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:36.000643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e89a39-a939-4f63-b2c6-2b1fee836c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:36.001566 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e89a39-a939-4f63-b2c6-2b1fee836c56 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:36.001841 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e89a39-a939-4f63-b2c6-2b1fee836c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:36.002130 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e89a39-a939-4f63-b2c6-2b1fee836c56 admin admin] PATH_INFO: `/v3/projects/310d60ed6fd445e2a66b059ddb8426cb/users/c67bd465afd143568d0287598eb6f435/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:36.019270 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e89a39-a939-4f63-b2c6-2b1fee836c56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=310d60ed6fd445e2a66b059ddb8426cb, user_id=c67bd465afd143568d0287598eb6f435, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:36.022206 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e89a39-a939-4f63-b2c6-2b1fee836c56 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:36.036443 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1303/2626] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:25:35 2022] PUT /identity/v3/projects/310d60ed6fd445e2a66b059ddb8426cb/users/c67bd465afd143568d0287598eb6f435/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:25:36.050812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef03a42-914a-476b-a207-2b90ded8450e None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:36.051195 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef03a42-914a-476b-a207-2b90ded8450e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:36.051538 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef03a42-914a-476b-a207-2b90ded8450e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:36.081817 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-1ef03a42-914a-476b-a207-2b90ded8450e None None] MFA Rules not processed for user `c67bd465afd143568d0287598eb6f435`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:25:36.083351 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-1ef03a42-914a-476b-a207-2b90ded8450e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:36.155467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1324/2627] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:25:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:25:36.171702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e89a39-a939-4f63-b2c6-2b1fee836c56 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:36.174177 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f0e89a39-a939-4f63-b2c6-2b1fee836c56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:36.233296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e89a39-a939-4f63-b2c6-2b1fee836c56 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:36.237633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f10276-f878-4626-ad1c-171970eb25a8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fd398c547554629927d2a80b3a8dee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:36.238270 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f10276-f878-4626-ad1c-171970eb25a8 service glance] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:36.238543 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f10276-f878-4626-ad1c-171970eb25a8 service glance] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:36.238778 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f10276-f878-4626-ad1c-171970eb25a8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:36.248050 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f10276-f878-4626-ad1c-171970eb25a8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:36.249505 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f10276-f878-4626-ad1c-171970eb25a8 service glance] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:36.258736 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1304/2628] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:25:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:25:52.361096 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef03a42-914a-476b-a207-2b90ded8450e None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:25:52.363373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-1ef03a42-914a-476b-a207-2b90ded8450e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:52.452314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef03a42-914a-476b-a207-2b90ded8450e None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:25:52.458106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3a9da-9054-4b39-a6df-2bbda6886320 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9e458c2c5f4ed1b3eddd84c2ec1703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:25:52.458754 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3a9da-9054-4b39-a6df-2bbda6886320 service placement] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:25:52.459041 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3a9da-9054-4b39-a6df-2bbda6886320 service placement] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:25:52.459305 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3a9da-9054-4b39-a6df-2bbda6886320 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:25:52.461190 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-fbc3a9da-9054-4b39-a6df-2bbda6886320 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:25:52.472806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc3a9da-9054-4b39-a6df-2bbda6886320 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:25:52.474371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc3a9da-9054-4b39-a6df-2bbda6886320 service placement] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:25:52.484060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1325/2629] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:25:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:26:00.370267 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f10276-f878-4626-ad1c-171970eb25a8 service glance] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:00.444524 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f10276-f878-4626-ad1c-171970eb25a8 service glance] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:00.450426 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445b84e-7177-479c-93a4-2e0e1cccb87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:00.450825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445b84e-7177-479c-93a4-2e0e1cccb87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:00.451095 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445b84e-7177-479c-93a4-2e0e1cccb87e admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:00.451334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445b84e-7177-479c-93a4-2e0e1cccb87e admin admin] PATH_INFO: `/v3/users/c67bd465afd143568d0287598eb6f435` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:00.452609 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445b84e-7177-479c-93a4-2e0e1cccb87e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c67bd465afd143568d0287598eb6f435)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:00.454541 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445b84e-7177-479c-93a4-2e0e1cccb87e admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:00.510757 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c445b84e-7177-479c-93a4-2e0e1cccb87e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c67bd465afd143568d0287598eb6f435'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:00.517605 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c445b84e-7177-479c-93a4-2e0e1cccb87e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c67bd465afd143568d0287598eb6f435'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:00.525899 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c445b84e-7177-479c-93a4-2e0e1cccb87e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c67bd465afd143568d0287598eb6f435'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:00.526525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c445b84e-7177-479c-93a4-2e0e1cccb87e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c67bd465afd143568d0287598eb6f435'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:00.527135 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-c445b84e-7177-479c-93a4-2e0e1cccb87e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c67bd465afd143568d0287598eb6f435'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:00.547495 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1305/2630] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:26:00 2022] DELETE /identity/v3/users/c67bd465afd143568d0287598eb6f435 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:00.552256 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c287fbe-3d40-488b-bf55-527211c19807 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:00.552822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c287fbe-3d40-488b-bf55-527211c19807 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:00.552822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c287fbe-3d40-488b-bf55-527211c19807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:00.570982 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-4c287fbe-3d40-488b-bf55-527211c19807 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:00.572191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-4c287fbe-3d40-488b-bf55-527211c19807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:00.634618 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1326/2631] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:00.644738 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445b84e-7177-479c-93a4-2e0e1cccb87e admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:00.646797 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-c445b84e-7177-479c-93a4-2e0e1cccb87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:00.713747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445b84e-7177-479c-93a4-2e0e1cccb87e admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:00.718900 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156606de-d946-4566-930f-3243547639be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:00.719554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156606de-d946-4566-930f-3243547639be service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:00.719922 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156606de-d946-4566-930f-3243547639be service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:00.720191 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156606de-d946-4566-930f-3243547639be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:00.731327 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156606de-d946-4566-930f-3243547639be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:00.732820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156606de-d946-4566-930f-3243547639be service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:00.742613 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1306/2632] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:26:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:26:01.027243 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c287fbe-3d40-488b-bf55-527211c19807 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:01.028789 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-4c287fbe-3d40-488b-bf55-527211c19807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:01.042710 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c287fbe-3d40-488b-bf55-527211c19807 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:01.048598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22516a-474e-42df-96df-96aa02ab90b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:01.049522 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22516a-474e-42df-96df-96aa02ab90b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:01.049788 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22516a-474e-42df-96df-96aa02ab90b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:01.050055 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22516a-474e-42df-96df-96aa02ab90b2 admin admin] PATH_INFO: `/v3/projects/310d60ed6fd445e2a66b059ddb8426cb` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:01.051485 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b22516a-474e-42df-96df-96aa02ab90b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=310d60ed6fd445e2a66b059ddb8426cb)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:01.053451 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b22516a-474e-42df-96df-96aa02ab90b2 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:01.105440 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-5b22516a-474e-42df-96df-96aa02ab90b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '310d60ed6fd445e2a66b059ddb8426cb', 'request_id': 'req-5b22516a-474e-42df-96df-96aa02ab90b2'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:01.111605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-5b22516a-474e-42df-96df-96aa02ab90b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '310d60ed6fd445e2a66b059ddb8426cb', 'request_id': 'req-5b22516a-474e-42df-96df-96aa02ab90b2'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:01.117692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-5b22516a-474e-42df-96df-96aa02ab90b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '310d60ed6fd445e2a66b059ddb8426cb', 'request_id': 'req-5b22516a-474e-42df-96df-96aa02ab90b2'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:01.132006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-5b22516a-474e-42df-96df-96aa02ab90b2 admin admin] The token cache is being invalidate because project 310d60ed6fd445e2a66b059ddb8426cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:26:01.132322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-5b22516a-474e-42df-96df-96aa02ab90b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:01.132978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-5b22516a-474e-42df-96df-96aa02ab90b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:01.134616 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1327/2633] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:26:01 2022] DELETE /identity/v3/projects/310d60ed6fd445e2a66b059ddb8426cb => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:01.139816 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945542f-eeb0-4abc-bc5b-a8071587dfb5 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:01.140100 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945542f-eeb0-4abc-bc5b-a8071587dfb5 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:01.140426 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945542f-eeb0-4abc-bc5b-a8071587dfb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:01.157984 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-5945542f-eeb0-4abc-bc5b-a8071587dfb5 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:01.159183 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-5945542f-eeb0-4abc-bc5b-a8071587dfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:01.216998 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1307/2634] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:01.225506 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756c928-1033-4247-a45e-8efe37bebebf None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:01.225766 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756c928-1033-4247-a45e-8efe37bebebf None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:01.226059 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756c928-1033-4247-a45e-8efe37bebebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:01.249590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-c756c928-1033-4247-a45e-8efe37bebebf None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:01.250999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c756c928-1033-4247-a45e-8efe37bebebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:01.260608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1328/2635] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:01.265572 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945542f-eeb0-4abc-bc5b-a8071587dfb5 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:01.278123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945542f-eeb0-4abc-bc5b-a8071587dfb5 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:01.283960 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a26b2-44ef-4dc4-9f5f-f930419efcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:01.284609 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a26b2-44ef-4dc4-9f5f-f930419efcda admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:01.284910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a26b2-44ef-4dc4-9f5f-f930419efcda admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:01.285182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a26b2-44ef-4dc4-9f5f-f930419efcda admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:01.285902 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a26b2-44ef-4dc4-9f5f-f930419efcda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 10:26:01.286199 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a26b2-44ef-4dc4-9f5f-f930419efcda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:01.287801 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a26b2-44ef-4dc4-9f5f-f930419efcda admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:01.301212 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1308/2636] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 10:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 10:26:01.305532 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c756c928-1033-4247-a45e-8efe37bebebf None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:01.315328 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c756c928-1033-4247-a45e-8efe37bebebf None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:01.321202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbee386-7e45-4db5-a9ab-e6f6133fd10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:01.321909 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbee386-7e45-4db5-a9ab-e6f6133fd10f admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:01.322181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbee386-7e45-4db5-a9ab-e6f6133fd10f admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:01.322434 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbee386-7e45-4db5-a9ab-e6f6133fd10f admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:01.323196 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbee386-7e45-4db5-a9ab-e6f6133fd10f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:01.325140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbee386-7e45-4db5-a9ab-e6f6133fd10f admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:01.353133 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1329/2637] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:26:01 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:26:01.357291 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a26b2-44ef-4dc4-9f5f-f930419efcda admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:01.367147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a26b2-44ef-4dc4-9f5f-f930419efcda admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:01.421772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65e549-f734-4455-b2bb-1f8dbc38cd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:01.422467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65e549-f734-4455-b2bb-1f8dbc38cd50 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:01.422767 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65e549-f734-4455-b2bb-1f8dbc38cd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:01.422920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65e549-f734-4455-b2bb-1f8dbc38cd50 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:01.423631 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65e549-f734-4455-b2bb-1f8dbc38cd50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:01.425568 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65e549-f734-4455-b2bb-1f8dbc38cd50 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:01.449382 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1309/2638] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:26:01 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:26:01.455418 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbee386-7e45-4db5-a9ab-e6f6133fd10f admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:01.467349 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbee386-7e45-4db5-a9ab-e6f6133fd10f admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:01.472313 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d83eeb-1bd4-4d47-bed2-d35e440dcd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:01.472993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d83eeb-1bd4-4d47-bed2-d35e440dcd8b admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:01.473265 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d83eeb-1bd4-4d47-bed2-d35e440dcd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:01.473533 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d83eeb-1bd4-4d47-bed2-d35e440dcd8b admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:01.474256 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d83eeb-1bd4-4d47-bed2-d35e440dcd8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:01.475874 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d83eeb-1bd4-4d47-bed2-d35e440dcd8b admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:01.486169 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1330/2639] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:01.492341 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65e549-f734-4455-b2bb-1f8dbc38cd50 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:01.505867 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65e549-f734-4455-b2bb-1f8dbc38cd50 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:01.512389 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a919a0-b6d2-450b-9a59-e2d9493a9786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:01.513079 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a919a0-b6d2-450b-9a59-e2d9493a9786 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:01.513356 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a919a0-b6d2-450b-9a59-e2d9493a9786 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:01.513610 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a919a0-b6d2-450b-9a59-e2d9493a9786 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:01.514351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a919a0-b6d2-450b-9a59-e2d9493a9786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:01.515940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a919a0-b6d2-450b-9a59-e2d9493a9786 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:01.526591 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1310/2640] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:01.530942 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d83eeb-1bd4-4d47-bed2-d35e440dcd8b admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:01.541859 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d83eeb-1bd4-4d47-bed2-d35e440dcd8b admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:01.547983 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ec277-f23a-4204-8935-6dc196d72e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:01.548907 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ec277-f23a-4204-8935-6dc196d72e1d admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:01.549184 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ec277-f23a-4204-8935-6dc196d72e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:01.549458 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ec277-f23a-4204-8935-6dc196d72e1d admin admin] PATH_INFO: `/v3/projects/88319789c26e4c7eb549e2e7e70093ee/users/cc05a8b1491f4555aa6574cd0904555d/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:01.565971 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ec277-f23a-4204-8935-6dc196d72e1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=88319789c26e4c7eb549e2e7e70093ee, user_id=cc05a8b1491f4555aa6574cd0904555d, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:01.568699 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ec277-f23a-4204-8935-6dc196d72e1d admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:01.583718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1331/2641] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:26:01 2022] PUT /identity/v3/projects/88319789c26e4c7eb549e2e7e70093ee/users/cc05a8b1491f4555aa6574cd0904555d/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:01.595747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29d626-e76e-4aa3-8677-3133ee423869 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:01.596309 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29d626-e76e-4aa3-8677-3133ee423869 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:01.596636 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29d626-e76e-4aa3-8677-3133ee423869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:01.628952 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-7f29d626-e76e-4aa3-8677-3133ee423869 None None] MFA Rules not processed for user `cc05a8b1491f4555aa6574cd0904555d`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:01.630163 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-7f29d626-e76e-4aa3-8677-3133ee423869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:01.701153 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1311/2642] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:01.713084 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ec277-f23a-4204-8935-6dc196d72e1d admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:01.714991 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-609ec277-f23a-4204-8935-6dc196d72e1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:01.773486 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ec277-f23a-4204-8935-6dc196d72e1d admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:01.777602 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f8e52-b24e-4698-b0dc-eaad6a8fcc32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:01.778250 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f8e52-b24e-4698-b0dc-eaad6a8fcc32 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:01.778525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f8e52-b24e-4698-b0dc-eaad6a8fcc32 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:01.778762 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f8e52-b24e-4698-b0dc-eaad6a8fcc32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:01.788949 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f8e52-b24e-4698-b0dc-eaad6a8fcc32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:01.790391 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f8e52-b24e-4698-b0dc-eaad6a8fcc32 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:01.800106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1332/2643] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:26:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:26:03.176956 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c2e1e-5439-4264-9e4a-61ef56b19208 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:03.177529 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c2e1e-5439-4264-9e4a-61ef56b19208 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:03.177529 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c2e1e-5439-4264-9e4a-61ef56b19208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:03.197084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-a22c2e1e-5439-4264-9e4a-61ef56b19208 None None] MFA Rules not processed for user `cc05a8b1491f4555aa6574cd0904555d`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:03.198324 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a22c2e1e-5439-4264-9e4a-61ef56b19208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:03.207175 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1312/2644] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:06.712764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f8e52-b24e-4698-b0dc-eaad6a8fcc32 service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:06.719796 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-d82f8e52-b24e-4698-b0dc-eaad6a8fcc32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:06.824951 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f8e52-b24e-4698-b0dc-eaad6a8fcc32 service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:06.831245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7b047-f5d3-47dd-9c5c-263f197f4742 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:06.831919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7b047-f5d3-47dd-9c5c-263f197f4742 service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:06.832203 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7b047-f5d3-47dd-9c5c-263f197f4742 service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:06.832825 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7b047-f5d3-47dd-9c5c-263f197f4742 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:06.834768 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-9fb7b047-f5d3-47dd-9c5c-263f197f4742 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:06.848362 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7b047-f5d3-47dd-9c5c-263f197f4742 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:06.850010 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7b047-f5d3-47dd-9c5c-263f197f4742 service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:06.870346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1333/2645] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:26:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:26:19.365288 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831d734-ec85-4d78-9c3b-f6862cfa34ef None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:19.365809 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831d734-ec85-4d78-9c3b-f6862cfa34ef None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:19.365809 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831d734-ec85-4d78-9c3b-f6862cfa34ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:19.431037 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-5831d734-ec85-4d78-9c3b-f6862cfa34ef None None] MFA Rules not processed for user `af62b2d3f73f482b8db4173af33752c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:19.432251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-5831d734-ec85-4d78-9c3b-f6862cfa34ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:19.514666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1313/2646] 10.222.0.26 () {58 vars in 1009 bytes} [Mon May 9 10:26:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:19.536256 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7b047-f5d3-47dd-9c5c-263f197f4742 service nova] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:19.537833 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-9fb7b047-f5d3-47dd-9c5c-263f197f4742 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:19.630815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7b047-f5d3-47dd-9c5c-263f197f4742 service nova] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:19.637341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e1dd8-bc6c-4d82-a74a-9991d56ba0c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380e04358f2349aa912d6a97518877cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:19.637984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e1dd8-bc6c-4d82-a74a-9991d56ba0c6 service swift] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:19.638261 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e1dd8-bc6c-4d82-a74a-9991d56ba0c6 service swift] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:19.638528 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e1dd8-bc6c-4d82-a74a-9991d56ba0c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:19.647193 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e1dd8-bc6c-4d82-a74a-9991d56ba0c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:19.648857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e1dd8-bc6c-4d82-a74a-9991d56ba0c6 service swift] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:19.656785 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1334/2647] 10.222.0.26 () {60 vars in 1384 bytes} [Mon May 9 10:26:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 10:26:20.235101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd7ba9-779b-4ec4-a195-6def45a801c3 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:20.235382 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd7ba9-779b-4ec4-a195-6def45a801c3 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:20.235649 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd7ba9-779b-4ec4-a195-6def45a801c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:20.254246 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-d2bd7ba9-779b-4ec4-a195-6def45a801c3 None None] MFA Rules not processed for user `af62b2d3f73f482b8db4173af33752c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:20.255493 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-d2bd7ba9-779b-4ec4-a195-6def45a801c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:20.263597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1314/2648] 10.222.0.26 () {58 vars in 967 bytes} [Mon May 9 10:26:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:20.276825 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e1dd8-bc6c-4d82-a74a-9991d56ba0c6 service swift] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:20.286956 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e1dd8-bc6c-4d82-a74a-9991d56ba0c6 service swift] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:20.292021 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d6cbb-b0bc-4afd-b073-b287b787bcc6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380e04358f2349aa912d6a97518877cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:20.292639 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d6cbb-b0bc-4afd-b073-b287b787bcc6 service swift] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:20.292943 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d6cbb-b0bc-4afd-b073-b287b787bcc6 service swift] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:20.293211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d6cbb-b0bc-4afd-b073-b287b787bcc6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:20.301733 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d6cbb-b0bc-4afd-b073-b287b787bcc6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:20.303350 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d6cbb-b0bc-4afd-b073-b287b787bcc6 service swift] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:20.311617 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1335/2649] 10.222.0.26 () {60 vars in 1384 bytes} [Mon May 9 10:26:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 10:26:20.525991 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07471d09-77cb-4945-85da-95102f380314 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:20.526522 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07471d09-77cb-4945-85da-95102f380314 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:20.526522 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07471d09-77cb-4945-85da-95102f380314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:20.546506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-07471d09-77cb-4945-85da-95102f380314 None None] MFA Rules not processed for user `af62b2d3f73f482b8db4173af33752c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:20.547864 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-07471d09-77cb-4945-85da-95102f380314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:20.556124 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1315/2650] 10.222.0.26 () {58 vars in 1009 bytes} [Mon May 9 10:26:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:20.568509 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d6cbb-b0bc-4afd-b073-b287b787bcc6 service swift] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:20.579217 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d6cbb-b0bc-4afd-b073-b287b787bcc6 service swift] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:20.585611 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e3496-a93a-469f-9965-c1031bc12fbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380e04358f2349aa912d6a97518877cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:20.586250 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e3496-a93a-469f-9965-c1031bc12fbf service swift] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:20.586524 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e3496-a93a-469f-9965-c1031bc12fbf service swift] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:20.586788 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e3496-a93a-469f-9965-c1031bc12fbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:20.595616 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e3496-a93a-469f-9965-c1031bc12fbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:20.597349 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e3496-a93a-469f-9965-c1031bc12fbf service swift] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:20.605538 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1336/2651] 10.222.0.26 () {60 vars in 1384 bytes} [Mon May 9 10:26:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 09 10:26:20.950145 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76e1e8-6c7c-4653-85fb-4eb9a8ba62ef None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:20.950427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76e1e8-6c7c-4653-85fb-4eb9a8ba62ef None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:20.950694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76e1e8-6c7c-4653-85fb-4eb9a8ba62ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:20.975034 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-5c76e1e8-6c7c-4653-85fb-4eb9a8ba62ef None None] MFA Rules not processed for user `af62b2d3f73f482b8db4173af33752c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:20.976224 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-5c76e1e8-6c7c-4653-85fb-4eb9a8ba62ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:20.984743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1316/2652] 10.222.0.26 () {58 vars in 967 bytes} [Mon May 9 10:26:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:21.000279 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e3496-a93a-469f-9965-c1031bc12fbf service swift] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:21.014980 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e3496-a93a-469f-9965-c1031bc12fbf service swift] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:21.021793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243525a3-227b-4f5a-b8fd-f84c61676605 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380e04358f2349aa912d6a97518877cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:21.022606 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243525a3-227b-4f5a-b8fd-f84c61676605 service swift] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:21.022929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243525a3-227b-4f5a-b8fd-f84c61676605 service swift] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:21.023148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243525a3-227b-4f5a-b8fd-f84c61676605 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:21.033414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243525a3-227b-4f5a-b8fd-f84c61676605 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:21.034727 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243525a3-227b-4f5a-b8fd-f84c61676605 service swift] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:21.043915 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1337/2653] 10.222.0.26 () {60 vars in 1384 bytes} [Mon May 9 10:26:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 10:26:23.233000 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76e1e8-6c7c-4653-85fb-4eb9a8ba62ef None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:23.320276 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76e1e8-6c7c-4653-85fb-4eb9a8ba62ef None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:23.328508 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870b658-060f-4684-878b-971791945fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:23.329415 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870b658-060f-4684-878b-971791945fe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:23.329697 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870b658-060f-4684-878b-971791945fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:23.329946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870b658-060f-4684-878b-971791945fe5 admin admin] PATH_INFO: `/v3/users/cc05a8b1491f4555aa6574cd0904555d` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:23.331268 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0870b658-060f-4684-878b-971791945fe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc05a8b1491f4555aa6574cd0904555d)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:23.333577 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0870b658-060f-4684-878b-971791945fe5 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:23.393932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0870b658-060f-4684-878b-971791945fe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc05a8b1491f4555aa6574cd0904555d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:23.404219 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0870b658-060f-4684-878b-971791945fe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc05a8b1491f4555aa6574cd0904555d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:23.413987 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0870b658-060f-4684-878b-971791945fe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc05a8b1491f4555aa6574cd0904555d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:23.414658 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0870b658-060f-4684-878b-971791945fe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc05a8b1491f4555aa6574cd0904555d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:23.415281 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0870b658-060f-4684-878b-971791945fe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc05a8b1491f4555aa6574cd0904555d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:23.432503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1317/2654] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:26:23 2022] DELETE /identity/v3/users/cc05a8b1491f4555aa6574cd0904555d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:23.437205 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9d58b-e9c9-440f-b269-e982b9673444 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:23.437711 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9d58b-e9c9-440f-b269-e982b9673444 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:23.437840 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9d58b-e9c9-440f-b269-e982b9673444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:23.455823 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-7df9d58b-e9c9-440f-b269-e982b9673444 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:23.457366 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-7df9d58b-e9c9-440f-b269-e982b9673444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:23.525060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1338/2655] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:23.537073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870b658-060f-4684-878b-971791945fe5 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:23.538705 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-0870b658-060f-4684-878b-971791945fe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:23.602326 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870b658-060f-4684-878b-971791945fe5 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:23.607591 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff983f3-4079-441c-bff7-108a4314ff42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:23.608358 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff983f3-4079-441c-bff7-108a4314ff42 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:23.608835 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff983f3-4079-441c-bff7-108a4314ff42 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:23.609256 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff983f3-4079-441c-bff7-108a4314ff42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:23.620235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff983f3-4079-441c-bff7-108a4314ff42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:23.621717 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff983f3-4079-441c-bff7-108a4314ff42 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:23.631782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1318/2656] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:26:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:26:23.843268 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9d58b-e9c9-440f-b269-e982b9673444 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:23.844926 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-7df9d58b-e9c9-440f-b269-e982b9673444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:23.859968 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9d58b-e9c9-440f-b269-e982b9673444 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:23.869042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c55b0a-e386-4018-89ae-63bbfa8ffb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:23.869943 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c55b0a-e386-4018-89ae-63bbfa8ffb58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:23.870220 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c55b0a-e386-4018-89ae-63bbfa8ffb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:23.870484 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c55b0a-e386-4018-89ae-63bbfa8ffb58 admin admin] PATH_INFO: `/v3/projects/88319789c26e4c7eb549e2e7e70093ee` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:23.871706 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c55b0a-e386-4018-89ae-63bbfa8ffb58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88319789c26e4c7eb549e2e7e70093ee)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:23.873688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c55b0a-e386-4018-89ae-63bbfa8ffb58 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:23.921439 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e5c55b0a-e386-4018-89ae-63bbfa8ffb58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88319789c26e4c7eb549e2e7e70093ee', 'request_id': 'req-e5c55b0a-e386-4018-89ae-63bbfa8ffb58'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:23.927550 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e5c55b0a-e386-4018-89ae-63bbfa8ffb58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88319789c26e4c7eb549e2e7e70093ee', 'request_id': 'req-e5c55b0a-e386-4018-89ae-63bbfa8ffb58'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:23.933211 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e5c55b0a-e386-4018-89ae-63bbfa8ffb58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88319789c26e4c7eb549e2e7e70093ee', 'request_id': 'req-e5c55b0a-e386-4018-89ae-63bbfa8ffb58'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:23.947791 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e5c55b0a-e386-4018-89ae-63bbfa8ffb58 admin admin] The token cache is being invalidate because project 88319789c26e4c7eb549e2e7e70093ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:26:23.948128 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e5c55b0a-e386-4018-89ae-63bbfa8ffb58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:23.948795 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e5c55b0a-e386-4018-89ae-63bbfa8ffb58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:23.950393 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1339/2657] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:26:23 2022] DELETE /identity/v3/projects/88319789c26e4c7eb549e2e7e70093ee => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:23.955946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c8b78-af3e-41e3-814b-aa9af9ccb131 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:23.956228 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c8b78-af3e-41e3-814b-aa9af9ccb131 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:23.956497 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c8b78-af3e-41e3-814b-aa9af9ccb131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:23.974532 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-0c5c8b78-af3e-41e3-814b-aa9af9ccb131 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:23.975744 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-0c5c8b78-af3e-41e3-814b-aa9af9ccb131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:24.037182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1319/2658] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 10:26:24.045516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976d2c0-5a44-4273-9680-0510267fd7b0 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:24.045821 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976d2c0-5a44-4273-9680-0510267fd7b0 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:24.046235 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976d2c0-5a44-4273-9680-0510267fd7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:24.067527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-1976d2c0-5a44-4273-9680-0510267fd7b0 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:24.068788 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-1976d2c0-5a44-4273-9680-0510267fd7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:24.079046 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1340/2659] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:24.083795 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c8b78-af3e-41e3-814b-aa9af9ccb131 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:24.093814 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c8b78-af3e-41e3-814b-aa9af9ccb131 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:24.099265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e44fc3-ed51-420c-8666-dab33aa88f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:24.099922 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e44fc3-ed51-420c-8666-dab33aa88f83 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:24.100198 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e44fc3-ed51-420c-8666-dab33aa88f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:24.100465 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e44fc3-ed51-420c-8666-dab33aa88f83 admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:24.101228 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e44fc3-ed51-420c-8666-dab33aa88f83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 10:26:24.101530 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e44fc3-ed51-420c-8666-dab33aa88f83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:24.103126 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e44fc3-ed51-420c-8666-dab33aa88f83 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:24.117315 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1320/2660] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 10:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 10:26:24.121717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976d2c0-5a44-4273-9680-0510267fd7b0 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:24.131621 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976d2c0-5a44-4273-9680-0510267fd7b0 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:24.136765 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa42deb-8d8e-478a-8514-d11bc0218ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:24.137399 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa42deb-8d8e-478a-8514-d11bc0218ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:24.137707 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa42deb-8d8e-478a-8514-d11bc0218ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:24.137983 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa42deb-8d8e-478a-8514-d11bc0218ca9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:24.138679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa42deb-8d8e-478a-8514-d11bc0218ca9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:24.140690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa42deb-8d8e-478a-8514-d11bc0218ca9 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:24.167946 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1341/2661] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:26:24 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:26:24.172361 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e44fc3-ed51-420c-8666-dab33aa88f83 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:24.182307 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e44fc3-ed51-420c-8666-dab33aa88f83 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:24.239869 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1d0b1-ac93-463d-94d6-6e8be4d040db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:24.240583 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1d0b1-ac93-463d-94d6-6e8be4d040db admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:24.240825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1d0b1-ac93-463d-94d6-6e8be4d040db admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:24.241070 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1d0b1-ac93-463d-94d6-6e8be4d040db admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:24.241758 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd1d0b1-ac93-463d-94d6-6e8be4d040db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:24.243688 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd1d0b1-ac93-463d-94d6-6e8be4d040db admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:24.266014 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1321/2662] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:26:24 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 10:26:24.270690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa42deb-8d8e-478a-8514-d11bc0218ca9 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:24.281356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa42deb-8d8e-478a-8514-d11bc0218ca9 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:24.286624 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b969bd5-03af-4542-aa7b-8f6a63afa20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:24.287336 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b969bd5-03af-4542-aa7b-8f6a63afa20f admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:24.287563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b969bd5-03af-4542-aa7b-8f6a63afa20f admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:24.287843 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b969bd5-03af-4542-aa7b-8f6a63afa20f admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:24.288628 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b969bd5-03af-4542-aa7b-8f6a63afa20f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:24.290337 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b969bd5-03af-4542-aa7b-8f6a63afa20f admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:24.302059 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1342/2663] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:24.308062 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1d0b1-ac93-463d-94d6-6e8be4d040db admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:24.321343 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1d0b1-ac93-463d-94d6-6e8be4d040db admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:24.328193 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65252a5-c309-462e-8308-91710552ad1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:24.328870 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65252a5-c309-462e-8308-91710552ad1c admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:24.329165 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65252a5-c309-462e-8308-91710552ad1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:24.329410 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65252a5-c309-462e-8308-91710552ad1c admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:24.330137 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65252a5-c309-462e-8308-91710552ad1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:24.331738 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65252a5-c309-462e-8308-91710552ad1c admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:24.342533 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1322/2664] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:24.347269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b969bd5-03af-4542-aa7b-8f6a63afa20f admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:24.385399 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b969bd5-03af-4542-aa7b-8f6a63afa20f admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:24.385399 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea5dc6-0fad-4a11-afe6-b581f6992872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:24.385399 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea5dc6-0fad-4a11-afe6-b581f6992872 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:24.385399 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea5dc6-0fad-4a11-afe6-b581f6992872 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:24.385902 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea5dc6-0fad-4a11-afe6-b581f6992872 admin admin] PATH_INFO: `/v3/projects/7b13e936fad64cbcafffb4f896606d4a/users/ffba3643a2b144739316ea0d5a4c01a9/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:24.395436 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea5dc6-0fad-4a11-afe6-b581f6992872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b13e936fad64cbcafffb4f896606d4a, user_id=ffba3643a2b144739316ea0d5a4c01a9, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:24.399295 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea5dc6-0fad-4a11-afe6-b581f6992872 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:24.416644 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1343/2665] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:26:24 2022] PUT /identity/v3/projects/7b13e936fad64cbcafffb4f896606d4a/users/ffba3643a2b144739316ea0d5a4c01a9/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:24.422123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1e6cd-70f3-49e9-bccd-93d84c4ed1bc None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:24.422409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1e6cd-70f3-49e9-bccd-93d84c4ed1bc None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:24.422665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1e6cd-70f3-49e9-bccd-93d84c4ed1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:24.440815 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-d8b1e6cd-70f3-49e9-bccd-93d84c4ed1bc None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:24.442009 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-d8b1e6cd-70f3-49e9-bccd-93d84c4ed1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:24.503176 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1323/2666] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:24.512908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea5dc6-0fad-4a11-afe6-b581f6992872 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:24.514783 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-d9ea5dc6-0fad-4a11-afe6-b581f6992872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:24.576239 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea5dc6-0fad-4a11-afe6-b581f6992872 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:24.580958 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b2c3a-fe46-4c9f-a87b-5dcf260dc743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:24.581606 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b2c3a-fe46-4c9f-a87b-5dcf260dc743 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:24.582184 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b2c3a-fe46-4c9f-a87b-5dcf260dc743 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:24.582459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b2c3a-fe46-4c9f-a87b-5dcf260dc743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:24.591937 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b2c3a-fe46-4c9f-a87b-5dcf260dc743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:24.593402 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b2c3a-fe46-4c9f-a87b-5dcf260dc743 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:24.603762 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1344/2667] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:26:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:26:28.995095 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9e283-be9c-4eba-9fad-f54cbfd12f6d None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:28.995615 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9e283-be9c-4eba-9fad-f54cbfd12f6d None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:28.995615 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9e283-be9c-4eba-9fad-f54cbfd12f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:29.037544 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-3cd9e283-be9c-4eba-9fad-f54cbfd12f6d None None] MFA Rules not processed for user `ffba3643a2b144739316ea0d5a4c01a9`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:29.038988 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-3cd9e283-be9c-4eba-9fad-f54cbfd12f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:29.135850 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1324/2668] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:29.136017 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b2c3a-fe46-4c9f-a87b-5dcf260dc743 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:29.145454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b2c3a-fe46-4c9f-a87b-5dcf260dc743 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:29.151370 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2576b82-d7c8-48a6-a195-18b8e24651f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:29.152373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2576b82-d7c8-48a6-a195-18b8e24651f3 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:29.152653 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2576b82-d7c8-48a6-a195-18b8e24651f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:29.152966 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2576b82-d7c8-48a6-a195-18b8e24651f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:29.153676 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2576b82-d7c8-48a6-a195-18b8e24651f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:29.156377 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2576b82-d7c8-48a6-a195-18b8e24651f3 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:29.187516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1345/2669] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:26:29 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:26:29.191200 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9e283-be9c-4eba-9fad-f54cbfd12f6d None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:29.202866 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9e283-be9c-4eba-9fad-f54cbfd12f6d None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:29.260498 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58781240-8fdd-42e2-9c00-633088a37bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:29.261200 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58781240-8fdd-42e2-9c00-633088a37bde admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:29.261479 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58781240-8fdd-42e2-9c00-633088a37bde admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:29.261747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58781240-8fdd-42e2-9c00-633088a37bde admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:29.262454 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58781240-8fdd-42e2-9c00-633088a37bde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:29.264398 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58781240-8fdd-42e2-9c00-633088a37bde admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:29.286772 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1325/2670] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:26:29 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 10:26:29.290743 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2576b82-d7c8-48a6-a195-18b8e24651f3 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:29.301880 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2576b82-d7c8-48a6-a195-18b8e24651f3 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:29.306518 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b77e0-0086-4c56-8267-8ff93c6302f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:29.307411 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b77e0-0086-4c56-8267-8ff93c6302f2 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:29.307686 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b77e0-0086-4c56-8267-8ff93c6302f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:29.307952 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b77e0-0086-4c56-8267-8ff93c6302f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:29.308651 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b77e0-0086-4c56-8267-8ff93c6302f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:29.310275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b77e0-0086-4c56-8267-8ff93c6302f2 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:29.321988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1346/2671] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 10:26:29.325769 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58781240-8fdd-42e2-9c00-633088a37bde admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:29.336246 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58781240-8fdd-42e2-9c00-633088a37bde admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:29.340852 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61a630-d1c5-42ab-863e-fab08eaae63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:29.341756 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61a630-d1c5-42ab-863e-fab08eaae63c admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:29.341999 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61a630-d1c5-42ab-863e-fab08eaae63c admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:29.342261 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61a630-d1c5-42ab-863e-fab08eaae63c admin admin] PATH_INFO: `/v3/projects/639896afc10345d78ebf3d6e239431f7/users/22cce7d2e3a04d60bad56d72ad6e152d/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:29.359661 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61a630-d1c5-42ab-863e-fab08eaae63c admin admin] RBAC: Authorizing `identity:create_grant(project_id=639896afc10345d78ebf3d6e239431f7, user_id=22cce7d2e3a04d60bad56d72ad6e152d, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:29.362342 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61a630-d1c5-42ab-863e-fab08eaae63c admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:29.390578 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1326/2672] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:26:29 2022] PUT /identity/v3/projects/639896afc10345d78ebf3d6e239431f7/users/22cce7d2e3a04d60bad56d72ad6e152d/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 10:26:29.396206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b77e0-0086-4c56-8267-8ff93c6302f2 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:29.407669 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b77e0-0086-4c56-8267-8ff93c6302f2 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:29.464513 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1071e3-068a-4423-963a-260ec43692dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:29.465215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1071e3-068a-4423-963a-260ec43692dc admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:29.465489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1071e3-068a-4423-963a-260ec43692dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:29.465910 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1071e3-068a-4423-963a-260ec43692dc admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:29.466491 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1071e3-068a-4423-963a-260ec43692dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:29.468106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1071e3-068a-4423-963a-260ec43692dc admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:29.478786 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1347/2673] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:29.482696 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61a630-d1c5-42ab-863e-fab08eaae63c admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:29.493182 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61a630-d1c5-42ab-863e-fab08eaae63c admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:29.498647 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329212db-40ca-4c30-9d2e-607880fdcaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:29.499710 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329212db-40ca-4c30-9d2e-607880fdcaad admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:29.500056 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329212db-40ca-4c30-9d2e-607880fdcaad admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:29.500326 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329212db-40ca-4c30-9d2e-607880fdcaad admin admin] PATH_INFO: `/v3/projects/639896afc10345d78ebf3d6e239431f7/users/22cce7d2e3a04d60bad56d72ad6e152d/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:29.503997 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329212db-40ca-4c30-9d2e-607880fdcaad admin admin] RBAC: Authorizing `identity:create_grant(project_id=639896afc10345d78ebf3d6e239431f7, user_id=22cce7d2e3a04d60bad56d72ad6e152d, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:29.506788 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329212db-40ca-4c30-9d2e-607880fdcaad admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:29.518025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1327/2674] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:26:29 2022] PUT /identity/v3/projects/639896afc10345d78ebf3d6e239431f7/users/22cce7d2e3a04d60bad56d72ad6e152d/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:35.527053 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539bef1-e977-4ab3-9279-35aff7a87ef0 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:35.527621 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539bef1-e977-4ab3-9279-35aff7a87ef0 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:35.527621 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539bef1-e977-4ab3-9279-35aff7a87ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:35.563503 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-4539bef1-e977-4ab3-9279-35aff7a87ef0 None None] MFA Rules not processed for user `22cce7d2e3a04d60bad56d72ad6e152d`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:35.564809 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-4539bef1-e977-4ab3-9279-35aff7a87ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:35.647143 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1348/2675] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:35 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:35.660136 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329212db-40ca-4c30-9d2e-607880fdcaad admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:35.662176 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-329212db-40ca-4c30-9d2e-607880fdcaad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:35.742123 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329212db-40ca-4c30-9d2e-607880fdcaad admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:35.750092 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127270d3-b294-4fc0-b614-d8f1b68a4fd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:35.750724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127270d3-b294-4fc0-b614-d8f1b68a4fd9 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:35.751000 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127270d3-b294-4fc0-b614-d8f1b68a4fd9 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:35.751265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127270d3-b294-4fc0-b614-d8f1b68a4fd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:35.810169 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127270d3-b294-4fc0-b614-d8f1b68a4fd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:35.811763 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127270d3-b294-4fc0-b614-d8f1b68a4fd9 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:35.821574 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1328/2676] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:26:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:26:37.143395 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54372ea4-ff7f-4a2b-8d84-5eb26dff22cc None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:37.143862 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54372ea4-ff7f-4a2b-8d84-5eb26dff22cc None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:37.143932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54372ea4-ff7f-4a2b-8d84-5eb26dff22cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:37.165465 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-54372ea4-ff7f-4a2b-8d84-5eb26dff22cc None None] MFA Rules not processed for user `ffba3643a2b144739316ea0d5a4c01a9`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:37.166880 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-54372ea4-ff7f-4a2b-8d84-5eb26dff22cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:37.178354 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1349/2677] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:37.385530 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127270d3-b294-4fc0-b614-d8f1b68a4fd9 service nova] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:37.387056 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-127270d3-b294-4fc0-b614-d8f1b68a4fd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:37.447780 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127270d3-b294-4fc0-b614-d8f1b68a4fd9 service nova] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:37.459793 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644c497-95dc-497e-8499-b35375c4cac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:37.460472 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644c497-95dc-497e-8499-b35375c4cac1 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:37.460851 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644c497-95dc-497e-8499-b35375c4cac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:37.461129 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644c497-95dc-497e-8499-b35375c4cac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:37.464725 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-8644c497-95dc-497e-8499-b35375c4cac1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:37.576601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644c497-95dc-497e-8499-b35375c4cac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:37.578474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644c497-95dc-497e-8499-b35375c4cac1 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:37.589992 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1329/2678] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:26:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:26:40.936943 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54372ea4-ff7f-4a2b-8d84-5eb26dff22cc None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:40.938591 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-54372ea4-ff7f-4a2b-8d84-5eb26dff22cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:40.956343 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54372ea4-ff7f-4a2b-8d84-5eb26dff22cc None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:40.961245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c864d-e699-4e5a-a9f7-ebf11d79a528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:40.962151 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c864d-e699-4e5a-a9f7-ebf11d79a528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:40.962423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c864d-e699-4e5a-a9f7-ebf11d79a528 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:40.962698 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c864d-e699-4e5a-a9f7-ebf11d79a528 admin admin] PATH_INFO: `/v3/users/76dc34045544496ab030864c5a0ffbff` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:40.977642 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c864d-e699-4e5a-a9f7-ebf11d79a528 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76dc34045544496ab030864c5a0ffbff)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:40.979808 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c864d-e699-4e5a-a9f7-ebf11d79a528 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:41.034467 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e00c864d-e699-4e5a-a9f7-ebf11d79a528 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76dc34045544496ab030864c5a0ffbff'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:41.041973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e00c864d-e699-4e5a-a9f7-ebf11d79a528 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76dc34045544496ab030864c5a0ffbff'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:41.042735 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e00c864d-e699-4e5a-a9f7-ebf11d79a528 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76dc34045544496ab030864c5a0ffbff'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:41.050716 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e00c864d-e699-4e5a-a9f7-ebf11d79a528 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76dc34045544496ab030864c5a0ffbff'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:41.051353 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e00c864d-e699-4e5a-a9f7-ebf11d79a528 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76dc34045544496ab030864c5a0ffbff'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:41.072539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1350/2679] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:26:40 2022] DELETE /identity/v3/users/76dc34045544496ab030864c5a0ffbff => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:41.281794 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644c497-95dc-497e-8499-b35375c4cac1 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:41.292812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644c497-95dc-497e-8499-b35375c4cac1 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:41.359354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db774b1-8329-4b99-b34e-c9d3c696071d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:41.360363 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db774b1-8329-4b99-b34e-c9d3c696071d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:41.360506 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db774b1-8329-4b99-b34e-c9d3c696071d admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:41.360817 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db774b1-8329-4b99-b34e-c9d3c696071d admin admin] PATH_INFO: `/v3/projects/8c91ae2e41a9487a9ae5fd4bf1b6d4f8` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:41.362077 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db774b1-8329-4b99-b34e-c9d3c696071d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c91ae2e41a9487a9ae5fd4bf1b6d4f8)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:41.364192 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db774b1-8329-4b99-b34e-c9d3c696071d admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:41.420210 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6db774b1-8329-4b99-b34e-c9d3c696071d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c91ae2e41a9487a9ae5fd4bf1b6d4f8', 'request_id': 'req-6db774b1-8329-4b99-b34e-c9d3c696071d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:41.427465 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6db774b1-8329-4b99-b34e-c9d3c696071d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c91ae2e41a9487a9ae5fd4bf1b6d4f8', 'request_id': 'req-6db774b1-8329-4b99-b34e-c9d3c696071d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:41.433533 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6db774b1-8329-4b99-b34e-c9d3c696071d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c91ae2e41a9487a9ae5fd4bf1b6d4f8', 'request_id': 'req-6db774b1-8329-4b99-b34e-c9d3c696071d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:41.446886 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6db774b1-8329-4b99-b34e-c9d3c696071d admin admin] The token cache is being invalidate because project 8c91ae2e41a9487a9ae5fd4bf1b6d4f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:26:41.447174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6db774b1-8329-4b99-b34e-c9d3c696071d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:41.447813 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6db774b1-8329-4b99-b34e-c9d3c696071d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:41.449430 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1330/2680] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:26:41 2022] DELETE /identity/v3/projects/8c91ae2e41a9487a9ae5fd4bf1b6d4f8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 10:26:41.454512 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37e15f-8557-4351-8ab7-1962022320a5 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:41.454803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37e15f-8557-4351-8ab7-1962022320a5 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:41.455089 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37e15f-8557-4351-8ab7-1962022320a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:41.479995 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-ba37e15f-8557-4351-8ab7-1962022320a5 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:41.481262 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-ba37e15f-8557-4351-8ab7-1962022320a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:41.542684 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1351/2681] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:41.550754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c090f4d-2f48-47cd-8226-af513a25ca60 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:41.551013 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c090f4d-2f48-47cd-8226-af513a25ca60 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:41.551310 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c090f4d-2f48-47cd-8226-af513a25ca60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:41.571906 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-7c090f4d-2f48-47cd-8226-af513a25ca60 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:41.573340 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-7c090f4d-2f48-47cd-8226-af513a25ca60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:41.582284 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1331/2682] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:41.586629 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37e15f-8557-4351-8ab7-1962022320a5 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:41.596599 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37e15f-8557-4351-8ab7-1962022320a5 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:41.601601 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac10ed-f919-4279-92e8-cd86bdb230b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:41.602383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac10ed-f919-4279-92e8-cd86bdb230b5 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:41.602511 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac10ed-f919-4279-92e8-cd86bdb230b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:41.602787 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac10ed-f919-4279-92e8-cd86bdb230b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:41.603515 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac10ed-f919-4279-92e8-cd86bdb230b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 10:26:41.603819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac10ed-f919-4279-92e8-cd86bdb230b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:41.605494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac10ed-f919-4279-92e8-cd86bdb230b5 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:41.618610 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1352/2683] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 10:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 10:26:41.623154 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c090f4d-2f48-47cd-8226-af513a25ca60 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:41.632923 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c090f4d-2f48-47cd-8226-af513a25ca60 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:41.638877 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9caa78-50e4-4c6c-82d9-568d5de0aa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:41.639570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9caa78-50e4-4c6c-82d9-568d5de0aa7b admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:41.639845 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9caa78-50e4-4c6c-82d9-568d5de0aa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:41.640117 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9caa78-50e4-4c6c-82d9-568d5de0aa7b admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:41.640855 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9caa78-50e4-4c6c-82d9-568d5de0aa7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:41.642766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9caa78-50e4-4c6c-82d9-568d5de0aa7b admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:41.671389 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1332/2684] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:26:41 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:26:41.675584 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac10ed-f919-4279-92e8-cd86bdb230b5 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:41.686330 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac10ed-f919-4279-92e8-cd86bdb230b5 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:41.746557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359be0c-6914-49d0-af62-4858ccd562be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:41.747231 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359be0c-6914-49d0-af62-4858ccd562be admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:41.747566 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359be0c-6914-49d0-af62-4858ccd562be admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:41.747929 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359be0c-6914-49d0-af62-4858ccd562be admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:41.748582 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a359be0c-6914-49d0-af62-4858ccd562be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:41.750695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a359be0c-6914-49d0-af62-4858ccd562be admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:41.772445 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1353/2685] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:26:41 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:26:41.777222 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9caa78-50e4-4c6c-82d9-568d5de0aa7b admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:41.793861 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9caa78-50e4-4c6c-82d9-568d5de0aa7b admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:41.801598 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5499450-0a90-4067-acc7-be11f932608e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:41.802541 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5499450-0a90-4067-acc7-be11f932608e admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:41.802980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5499450-0a90-4067-acc7-be11f932608e admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:41.803337 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5499450-0a90-4067-acc7-be11f932608e admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:41.804237 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5499450-0a90-4067-acc7-be11f932608e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:41.806498 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5499450-0a90-4067-acc7-be11f932608e admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:41.820094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1333/2686] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:41.825136 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359be0c-6914-49d0-af62-4858ccd562be admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:41.836156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359be0c-6914-49d0-af62-4858ccd562be admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:41.842073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319005b-eb5d-475e-ab81-fbe55bea054a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:41.842788 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319005b-eb5d-475e-ab81-fbe55bea054a admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:41.843194 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319005b-eb5d-475e-ab81-fbe55bea054a admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:41.843468 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319005b-eb5d-475e-ab81-fbe55bea054a admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:41.844408 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319005b-eb5d-475e-ab81-fbe55bea054a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:41.846240 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319005b-eb5d-475e-ab81-fbe55bea054a admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:41.858329 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1354/2687] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:41.862951 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5499450-0a90-4067-acc7-be11f932608e admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:41.873516 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5499450-0a90-4067-acc7-be11f932608e admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:41.879080 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd69154-ed24-4a22-91dd-93fc07184fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:41.879958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd69154-ed24-4a22-91dd-93fc07184fc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:41.880258 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd69154-ed24-4a22-91dd-93fc07184fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:41.880521 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd69154-ed24-4a22-91dd-93fc07184fc9 admin admin] PATH_INFO: `/v3/projects/1951331ff5ab48c7b661c964d183170a/users/788ec3f733004182b6f0827b7662e226/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:41.897025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd69154-ed24-4a22-91dd-93fc07184fc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1951331ff5ab48c7b661c964d183170a, user_id=788ec3f733004182b6f0827b7662e226, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:41.899765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd69154-ed24-4a22-91dd-93fc07184fc9 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:41.916629 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1334/2688] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:26:41 2022] PUT /identity/v3/projects/1951331ff5ab48c7b661c964d183170a/users/788ec3f733004182b6f0827b7662e226/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:41.928202 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800685a-2f6a-464f-a414-e616a0a4b48b None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:41.928485 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800685a-2f6a-464f-a414-e616a0a4b48b None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:41.928787 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800685a-2f6a-464f-a414-e616a0a4b48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:41.960557 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-7800685a-2f6a-464f-a414-e616a0a4b48b None None] MFA Rules not processed for user `788ec3f733004182b6f0827b7662e226`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:41.961330 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-7800685a-2f6a-464f-a414-e616a0a4b48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:42.033019 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1355/2689] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:42.037503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd69154-ed24-4a22-91dd-93fc07184fc9 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:42.103103 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd69154-ed24-4a22-91dd-93fc07184fc9 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:42.108021 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aedd80-73ef-4386-9545-d18a99da1e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:42.108834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aedd80-73ef-4386-9545-d18a99da1e69 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:42.109099 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aedd80-73ef-4386-9545-d18a99da1e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:42.109369 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aedd80-73ef-4386-9545-d18a99da1e69 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:42.110075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aedd80-73ef-4386-9545-d18a99da1e69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:42.111993 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aedd80-73ef-4386-9545-d18a99da1e69 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:42.141069 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1335/2690] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:26:42 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:26:42.145253 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800685a-2f6a-464f-a414-e616a0a4b48b None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:42.156186 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800685a-2f6a-464f-a414-e616a0a4b48b None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:42.212163 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9936f-13f7-4d74-87a0-6595982fec47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:42.212873 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9936f-13f7-4d74-87a0-6595982fec47 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:42.213131 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9936f-13f7-4d74-87a0-6595982fec47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:42.213403 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9936f-13f7-4d74-87a0-6595982fec47 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:42.214128 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d9936f-13f7-4d74-87a0-6595982fec47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:42.216036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d9936f-13f7-4d74-87a0-6595982fec47 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:42.237610 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1356/2691] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:26:42 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:26:42.241776 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aedd80-73ef-4386-9545-d18a99da1e69 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:42.253888 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aedd80-73ef-4386-9545-d18a99da1e69 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:42.259433 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3452e8-d7a4-4fb2-b874-bf4ac7a85692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:42.260112 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3452e8-d7a4-4fb2-b874-bf4ac7a85692 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:42.260391 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3452e8-d7a4-4fb2-b874-bf4ac7a85692 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:42.260654 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3452e8-d7a4-4fb2-b874-bf4ac7a85692 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:42.261427 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3452e8-d7a4-4fb2-b874-bf4ac7a85692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:42.263053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3452e8-d7a4-4fb2-b874-bf4ac7a85692 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:42.274535 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1336/2692] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:42.278577 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9936f-13f7-4d74-87a0-6595982fec47 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:42.289021 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9936f-13f7-4d74-87a0-6595982fec47 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:42.294021 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d1223-227a-4517-8611-ad6de76030e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:42.294680 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d1223-227a-4517-8611-ad6de76030e0 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:42.294959 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d1223-227a-4517-8611-ad6de76030e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:42.295224 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d1223-227a-4517-8611-ad6de76030e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:42.296061 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d1223-227a-4517-8611-ad6de76030e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:42.298066 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d1223-227a-4517-8611-ad6de76030e0 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:42.314417 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1357/2693] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:42.318815 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3452e8-d7a4-4fb2-b874-bf4ac7a85692 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:42.329611 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3452e8-d7a4-4fb2-b874-bf4ac7a85692 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:42.335088 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b8d471-46f3-432c-aa19-50b7c0abac9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:42.335970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8d471-46f3-432c-aa19-50b7c0abac9a admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:42.336249 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8d471-46f3-432c-aa19-50b7c0abac9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:42.336530 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8d471-46f3-432c-aa19-50b7c0abac9a admin admin] PATH_INFO: `/v3/projects/c7279390c3164c5eae9259d33f67c772/users/257ec558f9cd44bdae25fc5310c3b2f4/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:42.353618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b8d471-46f3-432c-aa19-50b7c0abac9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7279390c3164c5eae9259d33f67c772, user_id=257ec558f9cd44bdae25fc5310c3b2f4, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:42.356195 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b8d471-46f3-432c-aa19-50b7c0abac9a admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:42.375527 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1337/2694] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:26:42 2022] PUT /identity/v3/projects/c7279390c3164c5eae9259d33f67c772/users/257ec558f9cd44bdae25fc5310c3b2f4/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 10:26:42.387178 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bec0ed-e944-48f0-aca1-df1a9ee17433 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:42.387547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bec0ed-e944-48f0-aca1-df1a9ee17433 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:42.387725 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bec0ed-e944-48f0-aca1-df1a9ee17433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:42.417232 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-c2bec0ed-e944-48f0-aca1-df1a9ee17433 None None] MFA Rules not processed for user `257ec558f9cd44bdae25fc5310c3b2f4`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:42.418458 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c2bec0ed-e944-48f0-aca1-df1a9ee17433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:42.488001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1358/2695] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:42.495815 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b8d471-46f3-432c-aa19-50b7c0abac9a admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:42.560273 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b8d471-46f3-432c-aa19-50b7c0abac9a admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:42.565106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a1da2-85fa-4d60-9a01-07974050f29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:42.565802 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a1da2-85fa-4d60-9a01-07974050f29b admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:42.566015 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a1da2-85fa-4d60-9a01-07974050f29b admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:42.566287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a1da2-85fa-4d60-9a01-07974050f29b admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:42.566977 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a1da2-85fa-4d60-9a01-07974050f29b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:42.568905 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a1da2-85fa-4d60-9a01-07974050f29b admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:42.598464 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1338/2696] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:26:42 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:26:42.602605 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bec0ed-e944-48f0-aca1-df1a9ee17433 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:42.613822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bec0ed-e944-48f0-aca1-df1a9ee17433 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:42.673426 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abca3c0-9ebb-4004-aae9-35d99557a41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:42.674116 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abca3c0-9ebb-4004-aae9-35d99557a41e admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:42.674393 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abca3c0-9ebb-4004-aae9-35d99557a41e admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:42.674662 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abca3c0-9ebb-4004-aae9-35d99557a41e admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:42.675494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abca3c0-9ebb-4004-aae9-35d99557a41e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:42.677871 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abca3c0-9ebb-4004-aae9-35d99557a41e admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:42.702611 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1359/2697] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:26:42 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:26:42.706975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a1da2-85fa-4d60-9a01-07974050f29b admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:42.718554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a1da2-85fa-4d60-9a01-07974050f29b admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:42.723409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b16a2-55f7-463f-bcb9-be5004a3af13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:42.724132 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b16a2-55f7-463f-bcb9-be5004a3af13 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:42.724372 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b16a2-55f7-463f-bcb9-be5004a3af13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:42.724640 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b16a2-55f7-463f-bcb9-be5004a3af13 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:42.725392 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b16a2-55f7-463f-bcb9-be5004a3af13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:42.727015 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b16a2-55f7-463f-bcb9-be5004a3af13 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:42.738194 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1339/2698] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:42.742608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abca3c0-9ebb-4004-aae9-35d99557a41e admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:42.753056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abca3c0-9ebb-4004-aae9-35d99557a41e admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:42.758448 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8eec50-d529-4a7e-84b3-a449019e8e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:42.759338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8eec50-d529-4a7e-84b3-a449019e8e00 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:42.759722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8eec50-d529-4a7e-84b3-a449019e8e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:42.759896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8eec50-d529-4a7e-84b3-a449019e8e00 admin admin] PATH_INFO: `/v3/projects/849bafaad207407c91bf82dbad48e04f/users/928a621911ab4283beab6f99215565e8/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:42.777728 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8eec50-d529-4a7e-84b3-a449019e8e00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=849bafaad207407c91bf82dbad48e04f, user_id=928a621911ab4283beab6f99215565e8, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:42.780413 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8eec50-d529-4a7e-84b3-a449019e8e00 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:42.795003 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1360/2699] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:26:42 2022] PUT /identity/v3/projects/849bafaad207407c91bf82dbad48e04f/users/928a621911ab4283beab6f99215565e8/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:42.799342 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b16a2-55f7-463f-bcb9-be5004a3af13 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:42.809657 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b16a2-55f7-463f-bcb9-be5004a3af13 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:42.872899 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d8ea3-470f-41fd-b25d-95e8ef1d30b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:42.873499 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d8ea3-470f-41fd-b25d-95e8ef1d30b7 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:42.873748 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d8ea3-470f-41fd-b25d-95e8ef1d30b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:42.874089 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d8ea3-470f-41fd-b25d-95e8ef1d30b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:42.874817 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d8ea3-470f-41fd-b25d-95e8ef1d30b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:42.876431 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d8ea3-470f-41fd-b25d-95e8ef1d30b7 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:42.887187 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1340/2700] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:42.891430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8eec50-d529-4a7e-84b3-a449019e8e00 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:42.902753 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8eec50-d529-4a7e-84b3-a449019e8e00 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:42.911838 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4f7c2-8e04-4a55-b5a9-21812652413b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:42.911838 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4f7c2-8e04-4a55-b5a9-21812652413b admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:42.911838 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4f7c2-8e04-4a55-b5a9-21812652413b admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:42.911838 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4f7c2-8e04-4a55-b5a9-21812652413b admin admin] PATH_INFO: `/v3/projects/849bafaad207407c91bf82dbad48e04f/users/928a621911ab4283beab6f99215565e8/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:42.912292 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4f7c2-8e04-4a55-b5a9-21812652413b admin admin] RBAC: Authorizing `identity:create_grant(project_id=849bafaad207407c91bf82dbad48e04f, user_id=928a621911ab4283beab6f99215565e8, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:42.914781 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4f7c2-8e04-4a55-b5a9-21812652413b admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:42.926580 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1361/2701] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:26:42 2022] PUT /identity/v3/projects/849bafaad207407c91bf82dbad48e04f/users/928a621911ab4283beab6f99215565e8/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:42.938942 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604a957-5226-426a-80a5-d20a7f21a4aa None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:42.939221 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604a957-5226-426a-80a5-d20a7f21a4aa None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:42.939527 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604a957-5226-426a-80a5-d20a7f21a4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:42.974284 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-9604a957-5226-426a-80a5-d20a7f21a4aa None None] MFA Rules not processed for user `928a621911ab4283beab6f99215565e8`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:42.975879 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-9604a957-5226-426a-80a5-d20a7f21a4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:43.061226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1341/2702] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:42 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:43.073301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4f7c2-8e04-4a55-b5a9-21812652413b admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:43.075521 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-21e4f7c2-8e04-4a55-b5a9-21812652413b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:43.142220 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4f7c2-8e04-4a55-b5a9-21812652413b admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:43.146154 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2da6c5-3618-4bdd-a45d-8372a5fec17e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:43.146797 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2da6c5-3618-4bdd-a45d-8372a5fec17e service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:43.147069 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2da6c5-3618-4bdd-a45d-8372a5fec17e service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:43.147338 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2da6c5-3618-4bdd-a45d-8372a5fec17e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:43.159156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2da6c5-3618-4bdd-a45d-8372a5fec17e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:43.160745 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2da6c5-3618-4bdd-a45d-8372a5fec17e service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:43.171464 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1362/2703] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:26:43 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 10:26:43.711904 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604a957-5226-426a-80a5-d20a7f21a4aa None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:43.722768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604a957-5226-426a-80a5-d20a7f21a4aa None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:43.728104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00fa175-eebe-4d21-b0ba-e638880ad102 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:43.728907 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00fa175-eebe-4d21-b0ba-e638880ad102 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:43.729275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00fa175-eebe-4d21-b0ba-e638880ad102 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:43.729572 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00fa175-eebe-4d21-b0ba-e638880ad102 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:43.797650 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00fa175-eebe-4d21-b0ba-e638880ad102 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:43.799106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00fa175-eebe-4d21-b0ba-e638880ad102 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:43.808911 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1342/2704] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:26:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:26:48.936746 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2da6c5-3618-4bdd-a45d-8372a5fec17e service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:48.938396 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-6d2da6c5-3618-4bdd-a45d-8372a5fec17e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:48.954621 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2da6c5-3618-4bdd-a45d-8372a5fec17e service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:48.959554 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d8ede-7ad4-4f23-89fb-3544a6971f9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:48.960178 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d8ede-7ad4-4f23-89fb-3544a6971f9e service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:48.960451 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d8ede-7ad4-4f23-89fb-3544a6971f9e service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:48.960720 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d8ede-7ad4-4f23-89fb-3544a6971f9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:49.017506 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d8ede-7ad4-4f23-89fb-3544a6971f9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:49.019012 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d8ede-7ad4-4f23-89fb-3544a6971f9e service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:49.029485 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1363/2705] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:26:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:26:51.127097 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00fa175-eebe-4d21-b0ba-e638880ad102 service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:51.194749 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00fa175-eebe-4d21-b0ba-e638880ad102 service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:51.199610 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c858bd-151e-4c2d-bdc9-9b19e0c79c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:51.200502 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c858bd-151e-4c2d-bdc9-9b19e0c79c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:51.200816 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c858bd-151e-4c2d-bdc9-9b19e0c79c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:51.201091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c858bd-151e-4c2d-bdc9-9b19e0c79c79 admin admin] PATH_INFO: `/v3/users/788ec3f733004182b6f0827b7662e226` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:51.202317 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c858bd-151e-4c2d-bdc9-9b19e0c79c79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=788ec3f733004182b6f0827b7662e226)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:51.204252 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c858bd-151e-4c2d-bdc9-9b19e0c79c79 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:51.252889 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-55c858bd-151e-4c2d-bdc9-9b19e0c79c79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '788ec3f733004182b6f0827b7662e226'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:51.260150 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-55c858bd-151e-4c2d-bdc9-9b19e0c79c79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '788ec3f733004182b6f0827b7662e226'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:51.268411 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-55c858bd-151e-4c2d-bdc9-9b19e0c79c79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '788ec3f733004182b6f0827b7662e226'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:51.269188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-55c858bd-151e-4c2d-bdc9-9b19e0c79c79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '788ec3f733004182b6f0827b7662e226'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:51.269788 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-55c858bd-151e-4c2d-bdc9-9b19e0c79c79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '788ec3f733004182b6f0827b7662e226'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:51.286384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1343/2706] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:26:51 2022] DELETE /identity/v3/users/788ec3f733004182b6f0827b7662e226 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:51.291086 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e8e46-900a-478f-85f0-da581c8fabe2 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:51.291601 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e8e46-900a-478f-85f0-da581c8fabe2 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:51.291693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e8e46-900a-478f-85f0-da581c8fabe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:51.310499 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-683e8e46-900a-478f-85f0-da581c8fabe2 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:51.311803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-683e8e46-900a-478f-85f0-da581c8fabe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:51.368852 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1364/2707] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:51.378849 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c858bd-151e-4c2d-bdc9-9b19e0c79c79 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:51.380334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-55c858bd-151e-4c2d-bdc9-9b19e0c79c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:51.441024 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c858bd-151e-4c2d-bdc9-9b19e0c79c79 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:51.445328 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16d27b-c591-4913-a7fa-46356244371a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:51.445974 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16d27b-c591-4913-a7fa-46356244371a service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:51.446251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16d27b-c591-4913-a7fa-46356244371a service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:51.446496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16d27b-c591-4913-a7fa-46356244371a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:51.462646 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16d27b-c591-4913-a7fa-46356244371a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:51.464093 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16d27b-c591-4913-a7fa-46356244371a service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:51.474400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1344/2708] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:26:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:26:51.754930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e8e46-900a-478f-85f0-da581c8fabe2 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:51.756415 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-683e8e46-900a-478f-85f0-da581c8fabe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:51.771540 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e8e46-900a-478f-85f0-da581c8fabe2 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:51.777325 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a856087-05d1-402c-bc45-a5390c7a829c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:51.778264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a856087-05d1-402c-bc45-a5390c7a829c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:51.778579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a856087-05d1-402c-bc45-a5390c7a829c admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:51.778834 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a856087-05d1-402c-bc45-a5390c7a829c admin admin] PATH_INFO: `/v3/projects/1951331ff5ab48c7b661c964d183170a` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:51.780274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a856087-05d1-402c-bc45-a5390c7a829c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1951331ff5ab48c7b661c964d183170a)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:51.782436 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a856087-05d1-402c-bc45-a5390c7a829c admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:51.837270 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8a856087-05d1-402c-bc45-a5390c7a829c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1951331ff5ab48c7b661c964d183170a', 'request_id': 'req-8a856087-05d1-402c-bc45-a5390c7a829c'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:51.843526 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8a856087-05d1-402c-bc45-a5390c7a829c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1951331ff5ab48c7b661c964d183170a', 'request_id': 'req-8a856087-05d1-402c-bc45-a5390c7a829c'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:51.849119 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8a856087-05d1-402c-bc45-a5390c7a829c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1951331ff5ab48c7b661c964d183170a', 'request_id': 'req-8a856087-05d1-402c-bc45-a5390c7a829c'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:51.868182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8a856087-05d1-402c-bc45-a5390c7a829c admin admin] The token cache is being invalidate because project 1951331ff5ab48c7b661c964d183170a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:26:51.868708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8a856087-05d1-402c-bc45-a5390c7a829c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:51.869860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8a856087-05d1-402c-bc45-a5390c7a829c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:51.872472 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1365/2709] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:26:51 2022] DELETE /identity/v3/projects/1951331ff5ab48c7b661c964d183170a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:51.878428 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16d27b-c591-4913-a7fa-46356244371a service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:51.892241 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16d27b-c591-4913-a7fa-46356244371a service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:51.951515 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3fa11-e3ab-429e-9a91-ee21a95cae9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:51.952540 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3fa11-e3ab-429e-9a91-ee21a95cae9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:51.952853 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3fa11-e3ab-429e-9a91-ee21a95cae9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:51.953131 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3fa11-e3ab-429e-9a91-ee21a95cae9d admin admin] PATH_INFO: `/v3/users/257ec558f9cd44bdae25fc5310c3b2f4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:51.954359 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3fa11-e3ab-429e-9a91-ee21a95cae9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=257ec558f9cd44bdae25fc5310c3b2f4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:51.956614 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3fa11-e3ab-429e-9a91-ee21a95cae9d admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:52.004283 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0bd3fa11-e3ab-429e-9a91-ee21a95cae9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '257ec558f9cd44bdae25fc5310c3b2f4'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:52.011194 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0bd3fa11-e3ab-429e-9a91-ee21a95cae9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '257ec558f9cd44bdae25fc5310c3b2f4'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:52.020332 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0bd3fa11-e3ab-429e-9a91-ee21a95cae9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '257ec558f9cd44bdae25fc5310c3b2f4'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:52.021003 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0bd3fa11-e3ab-429e-9a91-ee21a95cae9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '257ec558f9cd44bdae25fc5310c3b2f4'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:52.021639 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0bd3fa11-e3ab-429e-9a91-ee21a95cae9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '257ec558f9cd44bdae25fc5310c3b2f4'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:52.040591 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1345/2710] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:26:51 2022] DELETE /identity/v3/users/257ec558f9cd44bdae25fc5310c3b2f4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:52.333586 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a856087-05d1-402c-bc45-a5390c7a829c admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:52.335182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-8a856087-05d1-402c-bc45-a5390c7a829c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:52.350631 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a856087-05d1-402c-bc45-a5390c7a829c admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:52.426346 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631ea3b-dbcc-479e-91d2-654a0e7bcc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:52.427223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631ea3b-dbcc-479e-91d2-654a0e7bcc68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:52.427499 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631ea3b-dbcc-479e-91d2-654a0e7bcc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:52.427770 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631ea3b-dbcc-479e-91d2-654a0e7bcc68 admin admin] PATH_INFO: `/v3/projects/c7279390c3164c5eae9259d33f67c772` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:52.429011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631ea3b-dbcc-479e-91d2-654a0e7bcc68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7279390c3164c5eae9259d33f67c772)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:52.430906 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631ea3b-dbcc-479e-91d2-654a0e7bcc68 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:52.477255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-6631ea3b-dbcc-479e-91d2-654a0e7bcc68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7279390c3164c5eae9259d33f67c772', 'request_id': 'req-6631ea3b-dbcc-479e-91d2-654a0e7bcc68'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:52.483663 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-6631ea3b-dbcc-479e-91d2-654a0e7bcc68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7279390c3164c5eae9259d33f67c772', 'request_id': 'req-6631ea3b-dbcc-479e-91d2-654a0e7bcc68'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:52.488806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-6631ea3b-dbcc-479e-91d2-654a0e7bcc68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7279390c3164c5eae9259d33f67c772', 'request_id': 'req-6631ea3b-dbcc-479e-91d2-654a0e7bcc68'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:52.502801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-6631ea3b-dbcc-479e-91d2-654a0e7bcc68 admin admin] The token cache is being invalidate because project c7279390c3164c5eae9259d33f67c772 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:26:52.503069 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-6631ea3b-dbcc-479e-91d2-654a0e7bcc68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:52.503709 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-6631ea3b-dbcc-479e-91d2-654a0e7bcc68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:52.505856 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1366/2711] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:26:52 2022] DELETE /identity/v3/projects/c7279390c3164c5eae9259d33f67c772 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:52.509953 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3fa11-e3ab-429e-9a91-ee21a95cae9d admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:52.511674 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-0bd3fa11-e3ab-429e-9a91-ee21a95cae9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:52.526445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3fa11-e3ab-429e-9a91-ee21a95cae9d admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:52.586604 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426ff18-141e-4442-9eba-e953611b4ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:52.587477 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426ff18-141e-4442-9eba-e953611b4ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:52.587755 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426ff18-141e-4442-9eba-e953611b4ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:52.588024 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426ff18-141e-4442-9eba-e953611b4ce8 admin admin] PATH_INFO: `/v3/users/928a621911ab4283beab6f99215565e8` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:52.589360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7426ff18-141e-4442-9eba-e953611b4ce8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=928a621911ab4283beab6f99215565e8)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:52.591263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7426ff18-141e-4442-9eba-e953611b4ce8 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:52.640631 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-7426ff18-141e-4442-9eba-e953611b4ce8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '928a621911ab4283beab6f99215565e8'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:52.647201 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-7426ff18-141e-4442-9eba-e953611b4ce8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '928a621911ab4283beab6f99215565e8'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:52.655925 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-7426ff18-141e-4442-9eba-e953611b4ce8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '928a621911ab4283beab6f99215565e8'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:52.656487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-7426ff18-141e-4442-9eba-e953611b4ce8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '928a621911ab4283beab6f99215565e8'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:52.657271 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-7426ff18-141e-4442-9eba-e953611b4ce8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '928a621911ab4283beab6f99215565e8'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:52.675758 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1346/2712] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:26:52 2022] DELETE /identity/v3/users/928a621911ab4283beab6f99215565e8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 10:26:52.986650 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631ea3b-dbcc-479e-91d2-654a0e7bcc68 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:52.988166 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-6631ea3b-dbcc-479e-91d2-654a0e7bcc68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:53.009678 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631ea3b-dbcc-479e-91d2-654a0e7bcc68 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:53.071780 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5efd86-0b9b-4f4f-b239-1e63f231ba36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:53.072649 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5efd86-0b9b-4f4f-b239-1e63f231ba36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:53.072958 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5efd86-0b9b-4f4f-b239-1e63f231ba36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:53.073231 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5efd86-0b9b-4f4f-b239-1e63f231ba36 admin admin] PATH_INFO: `/v3/projects/849bafaad207407c91bf82dbad48e04f` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:53.074422 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5efd86-0b9b-4f4f-b239-1e63f231ba36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=849bafaad207407c91bf82dbad48e04f)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:53.076337 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5efd86-0b9b-4f4f-b239-1e63f231ba36 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:53.124578 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-de5efd86-0b9b-4f4f-b239-1e63f231ba36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '849bafaad207407c91bf82dbad48e04f', 'request_id': 'req-de5efd86-0b9b-4f4f-b239-1e63f231ba36'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:53.130165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-de5efd86-0b9b-4f4f-b239-1e63f231ba36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '849bafaad207407c91bf82dbad48e04f', 'request_id': 'req-de5efd86-0b9b-4f4f-b239-1e63f231ba36'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:53.137134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-de5efd86-0b9b-4f4f-b239-1e63f231ba36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '849bafaad207407c91bf82dbad48e04f', 'request_id': 'req-de5efd86-0b9b-4f4f-b239-1e63f231ba36'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:53.151878 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-de5efd86-0b9b-4f4f-b239-1e63f231ba36 admin admin] The token cache is being invalidate because project 849bafaad207407c91bf82dbad48e04f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:26:53.152161 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-de5efd86-0b9b-4f4f-b239-1e63f231ba36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:53.152806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-de5efd86-0b9b-4f4f-b239-1e63f231ba36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:53.154824 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1367/2713] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:26:52 2022] DELETE /identity/v3/projects/849bafaad207407c91bf82dbad48e04f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:53.160743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc20f89-c8c0-4081-be53-ae3b1f0067e1 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:53.161026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc20f89-c8c0-4081-be53-ae3b1f0067e1 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:53.161301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc20f89-c8c0-4081-be53-ae3b1f0067e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:53.180450 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-7bc20f89-c8c0-4081-be53-ae3b1f0067e1 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:53.181647 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-7bc20f89-c8c0-4081-be53-ae3b1f0067e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:53.247067 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1347/2714] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:53.255751 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec5612-a4b6-41c3-bfbd-14bab74f7a97 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:53.256034 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec5612-a4b6-41c3-bfbd-14bab74f7a97 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:53.256307 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec5612-a4b6-41c3-bfbd-14bab74f7a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:53.274724 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-65ec5612-a4b6-41c3-bfbd-14bab74f7a97 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:53.275942 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-65ec5612-a4b6-41c3-bfbd-14bab74f7a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:53.284248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1368/2715] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 10:26:53.288780 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc20f89-c8c0-4081-be53-ae3b1f0067e1 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:53.298718 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc20f89-c8c0-4081-be53-ae3b1f0067e1 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:53.304275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aaa1a0-4902-42da-a8a9-6f8928b396f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:53.305032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aaa1a0-4902-42da-a8a9-6f8928b396f7 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:53.305302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aaa1a0-4902-42da-a8a9-6f8928b396f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:53.305564 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aaa1a0-4902-42da-a8a9-6f8928b396f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:53.306288 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aaa1a0-4902-42da-a8a9-6f8928b396f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 10:26:53.306585 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aaa1a0-4902-42da-a8a9-6f8928b396f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:53.308177 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aaa1a0-4902-42da-a8a9-6f8928b396f7 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:53.321501 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1348/2716] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 10:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 10:26:53.328088 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec5612-a4b6-41c3-bfbd-14bab74f7a97 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:53.341936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec5612-a4b6-41c3-bfbd-14bab74f7a97 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:53.347485 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c7fed-d603-42aa-aa48-d453d3a18ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:53.348242 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c7fed-d603-42aa-aa48-d453d3a18ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:53.348497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c7fed-d603-42aa-aa48-d453d3a18ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:53.348824 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c7fed-d603-42aa-aa48-d453d3a18ae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:53.349510 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916c7fed-d603-42aa-aa48-d453d3a18ae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:53.351453 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916c7fed-d603-42aa-aa48-d453d3a18ae5 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:53.380027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1369/2717] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:26:53 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:26:53.384478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aaa1a0-4902-42da-a8a9-6f8928b396f7 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:53.394928 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aaa1a0-4902-42da-a8a9-6f8928b396f7 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:53.458338 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc485e5-e99c-4da2-9e83-534422106f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:53.459129 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc485e5-e99c-4da2-9e83-534422106f5f admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:53.459460 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc485e5-e99c-4da2-9e83-534422106f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:53.459790 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc485e5-e99c-4da2-9e83-534422106f5f admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:53.460886 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc485e5-e99c-4da2-9e83-534422106f5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:53.463775 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc485e5-e99c-4da2-9e83-534422106f5f admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:53.491278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1349/2718] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:26:53 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:26:53.495422 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c7fed-d603-42aa-aa48-d453d3a18ae5 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:53.506294 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c7fed-d603-42aa-aa48-d453d3a18ae5 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:53.511758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76772bee-b2f6-4916-b808-cb5008e208a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:53.512426 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76772bee-b2f6-4916-b808-cb5008e208a6 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:53.512702 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76772bee-b2f6-4916-b808-cb5008e208a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:53.513000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76772bee-b2f6-4916-b808-cb5008e208a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:53.514087 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76772bee-b2f6-4916-b808-cb5008e208a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:53.515800 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76772bee-b2f6-4916-b808-cb5008e208a6 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:53.526543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1370/2719] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:53.530639 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc485e5-e99c-4da2-9e83-534422106f5f admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:53.541318 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc485e5-e99c-4da2-9e83-534422106f5f admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:53.546694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42609cd-ca5e-4b91-bef9-05c07d68e126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:53.547364 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42609cd-ca5e-4b91-bef9-05c07d68e126 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:53.547639 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42609cd-ca5e-4b91-bef9-05c07d68e126 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:53.547912 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42609cd-ca5e-4b91-bef9-05c07d68e126 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:53.548679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42609cd-ca5e-4b91-bef9-05c07d68e126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:53.550338 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42609cd-ca5e-4b91-bef9-05c07d68e126 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:53.564252 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1350/2720] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:53.565992 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76772bee-b2f6-4916-b808-cb5008e208a6 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:53.576195 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76772bee-b2f6-4916-b808-cb5008e208a6 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:53.581924 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46cbd8c-3072-4936-9132-6e0aa30d4c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:53.582939 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cbd8c-3072-4936-9132-6e0aa30d4c8b admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:53.583213 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cbd8c-3072-4936-9132-6e0aa30d4c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:53.583481 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cbd8c-3072-4936-9132-6e0aa30d4c8b admin admin] PATH_INFO: `/v3/projects/470e2d4188924380a4ad961079798134/users/3478b8666e524804be438ab3288d8347/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:53.598670 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46cbd8c-3072-4936-9132-6e0aa30d4c8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=470e2d4188924380a4ad961079798134, user_id=3478b8666e524804be438ab3288d8347, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:53.601466 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46cbd8c-3072-4936-9132-6e0aa30d4c8b admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:53.615667 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1371/2721] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:26:53 2022] PUT /identity/v3/projects/470e2d4188924380a4ad961079798134/users/3478b8666e524804be438ab3288d8347/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:53.627258 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cb56f-f7af-4755-961a-50a4650bfb98 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:53.627529 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cb56f-f7af-4755-961a-50a4650bfb98 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:53.627815 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cb56f-f7af-4755-961a-50a4650bfb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:53.653770 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-347cb56f-f7af-4755-961a-50a4650bfb98 None None] MFA Rules not processed for user `3478b8666e524804be438ab3288d8347`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:53.655062 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-347cb56f-f7af-4755-961a-50a4650bfb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:53.725557 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1351/2722] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:53.731820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46cbd8c-3072-4936-9132-6e0aa30d4c8b admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:53.803789 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46cbd8c-3072-4936-9132-6e0aa30d4c8b admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:53.808518 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0960f1-5a68-4b3e-96e5-4f08bc61cfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:53.809223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0960f1-5a68-4b3e-96e5-4f08bc61cfd9 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:53.809503 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0960f1-5a68-4b3e-96e5-4f08bc61cfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:53.809759 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0960f1-5a68-4b3e-96e5-4f08bc61cfd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:53.810501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0960f1-5a68-4b3e-96e5-4f08bc61cfd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:53.812494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0960f1-5a68-4b3e-96e5-4f08bc61cfd9 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:53.850152 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1372/2723] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:26:53 2022] POST /identity/v3/projects => generated 381 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 10:26:53.854664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347cb56f-f7af-4755-961a-50a4650bfb98 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:53.867415 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347cb56f-f7af-4755-961a-50a4650bfb98 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:53.925707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9777f7-9a37-4f7e-9d8a-b340623e47df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:53.926401 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9777f7-9a37-4f7e-9d8a-b340623e47df admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:53.926676 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9777f7-9a37-4f7e-9d8a-b340623e47df admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:53.926948 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9777f7-9a37-4f7e-9d8a-b340623e47df admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:53.927642 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9777f7-9a37-4f7e-9d8a-b340623e47df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:53.929658 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9777f7-9a37-4f7e-9d8a-b340623e47df admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:53.949846 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1352/2724] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:26:53 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:26:53.953968 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0960f1-5a68-4b3e-96e5-4f08bc61cfd9 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:53.964867 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0960f1-5a68-4b3e-96e5-4f08bc61cfd9 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:53.969757 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9819df2-cf7e-4544-a62a-4032886c3500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:53.970410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9819df2-cf7e-4544-a62a-4032886c3500 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:53.970688 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9819df2-cf7e-4544-a62a-4032886c3500 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:53.970956 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9819df2-cf7e-4544-a62a-4032886c3500 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:53.971681 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9819df2-cf7e-4544-a62a-4032886c3500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:53.973347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9819df2-cf7e-4544-a62a-4032886c3500 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:53.984548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1373/2725] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:53.989293 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9777f7-9a37-4f7e-9d8a-b340623e47df admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:53.999703 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9777f7-9a37-4f7e-9d8a-b340623e47df admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:54.005277 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f48a5-d5bc-499d-8e44-2e0eb3fc4e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:54.006318 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f48a5-d5bc-499d-8e44-2e0eb3fc4e95 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:54.006713 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f48a5-d5bc-499d-8e44-2e0eb3fc4e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:54.006976 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f48a5-d5bc-499d-8e44-2e0eb3fc4e95 admin admin] PATH_INFO: `/v3/projects/8245a319eb514e838e199d831ae12963/users/b747555c12164ef18fec925df23cf108/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:54.024385 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f48a5-d5bc-499d-8e44-2e0eb3fc4e95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8245a319eb514e838e199d831ae12963, user_id=b747555c12164ef18fec925df23cf108, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:54.028268 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f48a5-d5bc-499d-8e44-2e0eb3fc4e95 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:54.044567 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1353/2726] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:26:53 2022] PUT /identity/v3/projects/8245a319eb514e838e199d831ae12963/users/b747555c12164ef18fec925df23cf108/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:54.048407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9819df2-cf7e-4544-a62a-4032886c3500 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:54.058544 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9819df2-cf7e-4544-a62a-4032886c3500 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:54.115368 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0ca7a-790e-4a7e-b27d-45accc360b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:54.116034 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0ca7a-790e-4a7e-b27d-45accc360b5b admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:54.116314 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0ca7a-790e-4a7e-b27d-45accc360b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:54.116583 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0ca7a-790e-4a7e-b27d-45accc360b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:54.117336 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0ca7a-790e-4a7e-b27d-45accc360b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:54.118933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0ca7a-790e-4a7e-b27d-45accc360b5b admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:54.128683 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1374/2727] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:54.132941 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f48a5-d5bc-499d-8e44-2e0eb3fc4e95 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:54.144033 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f48a5-d5bc-499d-8e44-2e0eb3fc4e95 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:54.148770 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe45d08-35e7-4ba1-95d6-b273bdb0d2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:54.149649 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe45d08-35e7-4ba1-95d6-b273bdb0d2e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:54.149924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe45d08-35e7-4ba1-95d6-b273bdb0d2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:54.150196 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe45d08-35e7-4ba1-95d6-b273bdb0d2e0 admin admin] PATH_INFO: `/v3/projects/8245a319eb514e838e199d831ae12963/users/b747555c12164ef18fec925df23cf108/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:54.152214 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe45d08-35e7-4ba1-95d6-b273bdb0d2e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8245a319eb514e838e199d831ae12963, user_id=b747555c12164ef18fec925df23cf108, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:54.154913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe45d08-35e7-4ba1-95d6-b273bdb0d2e0 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:54.165887 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1354/2728] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:26:54 2022] PUT /identity/v3/projects/8245a319eb514e838e199d831ae12963/users/b747555c12164ef18fec925df23cf108/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:54.177521 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ffa2e-713c-40e2-b2bb-2574d3a4ee6f None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:54.177809 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ffa2e-713c-40e2-b2bb-2574d3a4ee6f None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:54.178080 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ffa2e-713c-40e2-b2bb-2574d3a4ee6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:54.210429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-cf6ffa2e-713c-40e2-b2bb-2574d3a4ee6f None None] MFA Rules not processed for user `b747555c12164ef18fec925df23cf108`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:54.211647 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-cf6ffa2e-713c-40e2-b2bb-2574d3a4ee6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:54.292035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1375/2729] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:54 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:54.306838 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe45d08-35e7-4ba1-95d6-b273bdb0d2e0 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:54.308697 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-2fe45d08-35e7-4ba1-95d6-b273bdb0d2e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:54.376202 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe45d08-35e7-4ba1-95d6-b273bdb0d2e0 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:54.380861 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c83338-3595-4a7f-b8fa-2a84a81659e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:54.381497 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c83338-3595-4a7f-b8fa-2a84a81659e8 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:54.381768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c83338-3595-4a7f-b8fa-2a84a81659e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:54.382032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c83338-3595-4a7f-b8fa-2a84a81659e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:54.392206 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c83338-3595-4a7f-b8fa-2a84a81659e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:54.393689 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c83338-3595-4a7f-b8fa-2a84a81659e8 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:54.403931 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1355/2730] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:26:54 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:26:54.481510 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ffa2e-713c-40e2-b2bb-2574d3a4ee6f None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:54.493884 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ffa2e-713c-40e2-b2bb-2574d3a4ee6f None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:54.498890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59592e6d-cc67-4589-93fc-34d9990314e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:54.499541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59592e6d-cc67-4589-93fc-34d9990314e2 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:54.499814 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59592e6d-cc67-4589-93fc-34d9990314e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:54.500080 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59592e6d-cc67-4589-93fc-34d9990314e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:54.566106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59592e6d-cc67-4589-93fc-34d9990314e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:54.567687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59592e6d-cc67-4589-93fc-34d9990314e2 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:54.577515 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1376/2731] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:26:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:26:54.860643 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c83338-3595-4a7f-b8fa-2a84a81659e8 service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:54.926929 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c83338-3595-4a7f-b8fa-2a84a81659e8 service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:54.931755 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588cde98-92d1-45a4-85d6-129653408972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:54.932621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588cde98-92d1-45a4-85d6-129653408972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:54.932931 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588cde98-92d1-45a4-85d6-129653408972 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:54.933187 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588cde98-92d1-45a4-85d6-129653408972 admin admin] PATH_INFO: `/v3/users/3478b8666e524804be438ab3288d8347` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:54.934410 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588cde98-92d1-45a4-85d6-129653408972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3478b8666e524804be438ab3288d8347)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:54.936369 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588cde98-92d1-45a4-85d6-129653408972 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:54.987033 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-588cde98-92d1-45a4-85d6-129653408972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3478b8666e524804be438ab3288d8347'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:54.994378 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-588cde98-92d1-45a4-85d6-129653408972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3478b8666e524804be438ab3288d8347'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:55.005337 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-588cde98-92d1-45a4-85d6-129653408972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3478b8666e524804be438ab3288d8347'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:55.006263 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-588cde98-92d1-45a4-85d6-129653408972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3478b8666e524804be438ab3288d8347'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:55.007324 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-588cde98-92d1-45a4-85d6-129653408972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3478b8666e524804be438ab3288d8347'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:55.029604 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1356/2732] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:26:54 2022] DELETE /identity/v3/users/3478b8666e524804be438ab3288d8347 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:55.035273 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921af959-eae6-4c42-bd95-1acccb0306fc None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:55.035662 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921af959-eae6-4c42-bd95-1acccb0306fc None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:55.035994 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921af959-eae6-4c42-bd95-1acccb0306fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:55.057670 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-921af959-eae6-4c42-bd95-1acccb0306fc None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:55.058890 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-921af959-eae6-4c42-bd95-1acccb0306fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:55.129306 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1377/2733] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:55.141659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588cde98-92d1-45a4-85d6-129653408972 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:55.143188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-588cde98-92d1-45a4-85d6-129653408972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:55.202326 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588cde98-92d1-45a4-85d6-129653408972 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:55.206595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7be15-7992-4b93-aacf-840eb0b73620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:55.207270 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7be15-7992-4b93-aacf-840eb0b73620 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:55.207546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7be15-7992-4b93-aacf-840eb0b73620 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:55.208023 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7be15-7992-4b93-aacf-840eb0b73620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:55.217580 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe7be15-7992-4b93-aacf-840eb0b73620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:55.219029 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe7be15-7992-4b93-aacf-840eb0b73620 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:55.228498 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1357/2734] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:26:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:26:55.502393 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921af959-eae6-4c42-bd95-1acccb0306fc None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:55.503856 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-921af959-eae6-4c42-bd95-1acccb0306fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:55.516503 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921af959-eae6-4c42-bd95-1acccb0306fc None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:55.522204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b48193-2813-4b4b-a879-9183dda3477b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:55.523085 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b48193-2813-4b4b-a879-9183dda3477b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:55.523353 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b48193-2813-4b4b-a879-9183dda3477b admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:55.523620 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b48193-2813-4b4b-a879-9183dda3477b admin admin] PATH_INFO: `/v3/projects/470e2d4188924380a4ad961079798134` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:55.525027 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b48193-2813-4b4b-a879-9183dda3477b admin admin] RBAC: Authorizing `identity:delete_project(project_id=470e2d4188924380a4ad961079798134)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:55.526837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b48193-2813-4b4b-a879-9183dda3477b admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:55.574199 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-13b48193-2813-4b4b-a879-9183dda3477b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '470e2d4188924380a4ad961079798134', 'request_id': 'req-13b48193-2813-4b4b-a879-9183dda3477b'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:55.580546 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-13b48193-2813-4b4b-a879-9183dda3477b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '470e2d4188924380a4ad961079798134', 'request_id': 'req-13b48193-2813-4b4b-a879-9183dda3477b'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:55.586810 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-13b48193-2813-4b4b-a879-9183dda3477b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '470e2d4188924380a4ad961079798134', 'request_id': 'req-13b48193-2813-4b4b-a879-9183dda3477b'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:55.601296 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-13b48193-2813-4b4b-a879-9183dda3477b admin admin] The token cache is being invalidate because project 470e2d4188924380a4ad961079798134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:26:55.601578 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-13b48193-2813-4b4b-a879-9183dda3477b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:55.602191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-13b48193-2813-4b4b-a879-9183dda3477b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:55.603772 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1378/2735] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:26:55 2022] DELETE /identity/v3/projects/470e2d4188924380a4ad961079798134 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:55.608217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7be15-7992-4b93-aacf-840eb0b73620 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:55.617933 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7be15-7992-4b93-aacf-840eb0b73620 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:55.678976 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64dc8a-8250-418c-ae40-c9115bd8e731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:55.679862 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64dc8a-8250-418c-ae40-c9115bd8e731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:55.680111 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64dc8a-8250-418c-ae40-c9115bd8e731 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:55.680378 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64dc8a-8250-418c-ae40-c9115bd8e731 admin admin] PATH_INFO: `/v3/users/b747555c12164ef18fec925df23cf108` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:55.681641 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d64dc8a-8250-418c-ae40-c9115bd8e731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b747555c12164ef18fec925df23cf108)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:55.683641 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d64dc8a-8250-418c-ae40-c9115bd8e731 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:55.729718 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-5d64dc8a-8250-418c-ae40-c9115bd8e731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b747555c12164ef18fec925df23cf108'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:55.736273 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-5d64dc8a-8250-418c-ae40-c9115bd8e731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b747555c12164ef18fec925df23cf108'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:55.744231 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-5d64dc8a-8250-418c-ae40-c9115bd8e731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b747555c12164ef18fec925df23cf108'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:55.744881 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-5d64dc8a-8250-418c-ae40-c9115bd8e731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b747555c12164ef18fec925df23cf108'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:55.745536 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-5d64dc8a-8250-418c-ae40-c9115bd8e731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b747555c12164ef18fec925df23cf108'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:55.766636 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1358/2736] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:26:55 2022] DELETE /identity/v3/users/b747555c12164ef18fec925df23cf108 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 10:26:56.054389 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b48193-2813-4b4b-a879-9183dda3477b admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:56.056064 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-13b48193-2813-4b4b-a879-9183dda3477b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:56.071096 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b48193-2813-4b4b-a879-9183dda3477b admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:56.132152 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b982bf-74b7-4318-8fcf-63c1c25076de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:56.133062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b982bf-74b7-4318-8fcf-63c1c25076de admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:56.133347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b982bf-74b7-4318-8fcf-63c1c25076de admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:56.133621 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b982bf-74b7-4318-8fcf-63c1c25076de admin admin] PATH_INFO: `/v3/projects/8245a319eb514e838e199d831ae12963` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:56.134828 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b982bf-74b7-4318-8fcf-63c1c25076de admin admin] RBAC: Authorizing `identity:delete_project(project_id=8245a319eb514e838e199d831ae12963)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:56.136760 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b982bf-74b7-4318-8fcf-63c1c25076de admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:56.185362 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-75b982bf-74b7-4318-8fcf-63c1c25076de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8245a319eb514e838e199d831ae12963', 'request_id': 'req-75b982bf-74b7-4318-8fcf-63c1c25076de'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:56.191899 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-75b982bf-74b7-4318-8fcf-63c1c25076de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8245a319eb514e838e199d831ae12963', 'request_id': 'req-75b982bf-74b7-4318-8fcf-63c1c25076de'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:56.197547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-75b982bf-74b7-4318-8fcf-63c1c25076de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8245a319eb514e838e199d831ae12963', 'request_id': 'req-75b982bf-74b7-4318-8fcf-63c1c25076de'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:56.211520 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-75b982bf-74b7-4318-8fcf-63c1c25076de admin admin] The token cache is being invalidate because project 8245a319eb514e838e199d831ae12963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:26:56.211774 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-75b982bf-74b7-4318-8fcf-63c1c25076de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:56.212395 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-75b982bf-74b7-4318-8fcf-63c1c25076de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:56.214048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1379/2737] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:26:56 2022] DELETE /identity/v3/projects/8245a319eb514e838e199d831ae12963 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:56.219244 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba657c4d-80b4-41a0-85b7-6cc97258a9d5 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:56.219718 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba657c4d-80b4-41a0-85b7-6cc97258a9d5 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:56.219805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba657c4d-80b4-41a0-85b7-6cc97258a9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:56.237234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-ba657c4d-80b4-41a0-85b7-6cc97258a9d5 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:56.238449 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-ba657c4d-80b4-41a0-85b7-6cc97258a9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:56.300047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1359/2738] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:56.308157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910dd079-0fa2-44d9-8562-8cebe9a0a64b None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:56.308407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910dd079-0fa2-44d9-8562-8cebe9a0a64b None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:56.308730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910dd079-0fa2-44d9-8562-8cebe9a0a64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:56.332993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-910dd079-0fa2-44d9-8562-8cebe9a0a64b None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:56.334210 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-910dd079-0fa2-44d9-8562-8cebe9a0a64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:56.343383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1380/2739] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:56.347747 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba657c4d-80b4-41a0-85b7-6cc97258a9d5 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:56.358409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba657c4d-80b4-41a0-85b7-6cc97258a9d5 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:56.363647 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7807d-8c27-4bb4-9eac-4aed4a02565f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:56.364301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7807d-8c27-4bb4-9eac-4aed4a02565f admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:56.364572 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7807d-8c27-4bb4-9eac-4aed4a02565f admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:56.364924 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7807d-8c27-4bb4-9eac-4aed4a02565f admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:56.365655 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7807d-8c27-4bb4-9eac-4aed4a02565f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 10:26:56.365960 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7807d-8c27-4bb4-9eac-4aed4a02565f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:56.367564 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7807d-8c27-4bb4-9eac-4aed4a02565f admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:56.379234 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1360/2740] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 10:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 10:26:56.383753 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910dd079-0fa2-44d9-8562-8cebe9a0a64b None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:56.393406 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910dd079-0fa2-44d9-8562-8cebe9a0a64b None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:56.399464 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bc65b-5f64-4ce6-8cd7-9ec91c7defa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:56.400116 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bc65b-5f64-4ce6-8cd7-9ec91c7defa9 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:56.400388 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bc65b-5f64-4ce6-8cd7-9ec91c7defa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:56.400651 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bc65b-5f64-4ce6-8cd7-9ec91c7defa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:56.401371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314bc65b-5f64-4ce6-8cd7-9ec91c7defa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:56.403293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314bc65b-5f64-4ce6-8cd7-9ec91c7defa9 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:56.432498 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1381/2741] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:26:56 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:26:56.436732 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7807d-8c27-4bb4-9eac-4aed4a02565f admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:56.447268 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7807d-8c27-4bb4-9eac-4aed4a02565f admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:56.504824 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c76dc6-4102-4df8-85ac-a9785a3a3e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:56.505490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c76dc6-4102-4df8-85ac-a9785a3a3e99 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:56.505769 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c76dc6-4102-4df8-85ac-a9785a3a3e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:56.506037 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c76dc6-4102-4df8-85ac-a9785a3a3e99 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:56.506720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c76dc6-4102-4df8-85ac-a9785a3a3e99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:56.508627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c76dc6-4102-4df8-85ac-a9785a3a3e99 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:56.530053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1361/2742] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:26:56 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:26:56.535417 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bc65b-5f64-4ce6-8cd7-9ec91c7defa9 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:56.545989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bc65b-5f64-4ce6-8cd7-9ec91c7defa9 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:56.550924 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c54b06-7c83-4c7d-b6d6-d0c037b1f9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:56.551576 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c54b06-7c83-4c7d-b6d6-d0c037b1f9a1 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:56.551852 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c54b06-7c83-4c7d-b6d6-d0c037b1f9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:56.552147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c54b06-7c83-4c7d-b6d6-d0c037b1f9a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:56.552914 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c54b06-7c83-4c7d-b6d6-d0c037b1f9a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:56.554499 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c54b06-7c83-4c7d-b6d6-d0c037b1f9a1 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:56.564993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1382/2743] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:56.569368 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c76dc6-4102-4df8-85ac-a9785a3a3e99 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:56.579496 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c76dc6-4102-4df8-85ac-a9785a3a3e99 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:56.584744 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7177712-0789-4173-9df8-2dcb08907d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:56.585415 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7177712-0789-4173-9df8-2dcb08907d8f admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:56.585691 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7177712-0789-4173-9df8-2dcb08907d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:56.585960 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7177712-0789-4173-9df8-2dcb08907d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:56.586682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7177712-0789-4173-9df8-2dcb08907d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:56.588302 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7177712-0789-4173-9df8-2dcb08907d8f admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:56.598541 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1362/2744] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:56.602780 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c54b06-7c83-4c7d-b6d6-d0c037b1f9a1 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:56.613134 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c54b06-7c83-4c7d-b6d6-d0c037b1f9a1 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:56.618464 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c15ab-7c9b-4931-aecd-2d7ee9c6ec3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:56.619344 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c15ab-7c9b-4931-aecd-2d7ee9c6ec3d admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:56.619602 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c15ab-7c9b-4931-aecd-2d7ee9c6ec3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:56.619885 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c15ab-7c9b-4931-aecd-2d7ee9c6ec3d admin admin] PATH_INFO: `/v3/projects/19cb7b5e8b2e4e4ba678c65ae8fef5c5/users/ccab8859d5db4a139a398141d159d809/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:56.636065 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16c15ab-7c9b-4931-aecd-2d7ee9c6ec3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=19cb7b5e8b2e4e4ba678c65ae8fef5c5, user_id=ccab8859d5db4a139a398141d159d809, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:56.638766 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16c15ab-7c9b-4931-aecd-2d7ee9c6ec3d admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:56.653684 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1383/2745] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:26:56 2022] PUT /identity/v3/projects/19cb7b5e8b2e4e4ba678c65ae8fef5c5/users/ccab8859d5db4a139a398141d159d809/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:56.665536 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492de87-2eaf-470c-a616-701b24c69b40 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:56.665820 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492de87-2eaf-470c-a616-701b24c69b40 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:56.666094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492de87-2eaf-470c-a616-701b24c69b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:56.694128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-3492de87-2eaf-470c-a616-701b24c69b40 None None] MFA Rules not processed for user `ccab8859d5db4a139a398141d159d809`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:56.695252 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-3492de87-2eaf-470c-a616-701b24c69b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:56.762250 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1363/2746] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:56.773359 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c15ab-7c9b-4931-aecd-2d7ee9c6ec3d admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:56.775147 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-f16c15ab-7c9b-4931-aecd-2d7ee9c6ec3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:56.834073 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c15ab-7c9b-4931-aecd-2d7ee9c6ec3d admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:56.839534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc95801-4eca-47ee-925f-9b39e38b2521 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:56.840428 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc95801-4eca-47ee-925f-9b39e38b2521 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:56.840854 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc95801-4eca-47ee-925f-9b39e38b2521 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:56.841218 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc95801-4eca-47ee-925f-9b39e38b2521 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:56.851853 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc95801-4eca-47ee-925f-9b39e38b2521 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:56.853304 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc95801-4eca-47ee-925f-9b39e38b2521 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:56.863361 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1384/2747] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:26:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 10:26:56.884326 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492de87-2eaf-470c-a616-701b24c69b40 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:56.952940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492de87-2eaf-470c-a616-701b24c69b40 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:56.959341 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb076f39-6f85-40ea-9303-f99986350533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:56.960206 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb076f39-6f85-40ea-9303-f99986350533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:56.960479 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb076f39-6f85-40ea-9303-f99986350533 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:56.960773 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb076f39-6f85-40ea-9303-f99986350533 admin admin] PATH_INFO: `/v3/users/ccab8859d5db4a139a398141d159d809` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:56.962054 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb076f39-6f85-40ea-9303-f99986350533 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccab8859d5db4a139a398141d159d809)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:56.964240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb076f39-6f85-40ea-9303-f99986350533 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:57.021435 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-fb076f39-6f85-40ea-9303-f99986350533 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccab8859d5db4a139a398141d159d809'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:57.028861 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-fb076f39-6f85-40ea-9303-f99986350533 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccab8859d5db4a139a398141d159d809'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:57.036576 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-fb076f39-6f85-40ea-9303-f99986350533 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccab8859d5db4a139a398141d159d809'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:57.037248 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-fb076f39-6f85-40ea-9303-f99986350533 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccab8859d5db4a139a398141d159d809'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:57.038004 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-fb076f39-6f85-40ea-9303-f99986350533 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccab8859d5db4a139a398141d159d809'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:57.054413 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1364/2748] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:26:56 2022] DELETE /identity/v3/users/ccab8859d5db4a139a398141d159d809 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:57.060742 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b8b97-9443-4c35-a7c8-f8d9bdb2a48d None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:57.061085 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b8b97-9443-4c35-a7c8-f8d9bdb2a48d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:57.061471 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b8b97-9443-4c35-a7c8-f8d9bdb2a48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:57.085163 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-052b8b97-9443-4c35-a7c8-f8d9bdb2a48d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:57.086406 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-052b8b97-9443-4c35-a7c8-f8d9bdb2a48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:57.157203 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1385/2749] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:57.167345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb076f39-6f85-40ea-9303-f99986350533 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:57.169055 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-fb076f39-6f85-40ea-9303-f99986350533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:57.230807 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb076f39-6f85-40ea-9303-f99986350533 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:57.235518 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491311f8-7d5c-4f3c-a36f-a7d25dad8177 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:57.236176 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491311f8-7d5c-4f3c-a36f-a7d25dad8177 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:57.236451 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491311f8-7d5c-4f3c-a36f-a7d25dad8177 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:57.236818 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491311f8-7d5c-4f3c-a36f-a7d25dad8177 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:57.246974 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491311f8-7d5c-4f3c-a36f-a7d25dad8177 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:57.248420 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491311f8-7d5c-4f3c-a36f-a7d25dad8177 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:57.258346 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1365/2750] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:26:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:26:57.541766 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b8b97-9443-4c35-a7c8-f8d9bdb2a48d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:57.541766 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-052b8b97-9443-4c35-a7c8-f8d9bdb2a48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:57.553164 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b8b97-9443-4c35-a7c8-f8d9bdb2a48d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:57.570579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8891043d-b36d-4cc5-b81e-87e949217d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:57.571483 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891043d-b36d-4cc5-b81e-87e949217d66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:57.571757 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891043d-b36d-4cc5-b81e-87e949217d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:57.572030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891043d-b36d-4cc5-b81e-87e949217d66 admin admin] PATH_INFO: `/v3/projects/19cb7b5e8b2e4e4ba678c65ae8fef5c5` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:57.573466 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8891043d-b36d-4cc5-b81e-87e949217d66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19cb7b5e8b2e4e4ba678c65ae8fef5c5)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:57.575453 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8891043d-b36d-4cc5-b81e-87e949217d66 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:57.631740 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8891043d-b36d-4cc5-b81e-87e949217d66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19cb7b5e8b2e4e4ba678c65ae8fef5c5', 'request_id': 'req-8891043d-b36d-4cc5-b81e-87e949217d66'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:57.637733 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8891043d-b36d-4cc5-b81e-87e949217d66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19cb7b5e8b2e4e4ba678c65ae8fef5c5', 'request_id': 'req-8891043d-b36d-4cc5-b81e-87e949217d66'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:57.642987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8891043d-b36d-4cc5-b81e-87e949217d66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19cb7b5e8b2e4e4ba678c65ae8fef5c5', 'request_id': 'req-8891043d-b36d-4cc5-b81e-87e949217d66'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:57.658547 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8891043d-b36d-4cc5-b81e-87e949217d66 admin admin] The token cache is being invalidate because project 19cb7b5e8b2e4e4ba678c65ae8fef5c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:26:57.658825 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8891043d-b36d-4cc5-b81e-87e949217d66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:57.659551 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-8891043d-b36d-4cc5-b81e-87e949217d66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:26:57.661277 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1386/2751] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:26:57 2022] DELETE /identity/v3/projects/19cb7b5e8b2e4e4ba678c65ae8fef5c5 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:57.666273 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67debbd-dd9f-41c6-ab00-416f651f49b5 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:57.666549 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67debbd-dd9f-41c6-ab00-416f651f49b5 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:57.666812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67debbd-dd9f-41c6-ab00-416f651f49b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:57.684843 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-c67debbd-dd9f-41c6-ab00-416f651f49b5 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:57.686170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-c67debbd-dd9f-41c6-ab00-416f651f49b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:57.748509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1366/2752] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:57.811492 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee80d9-f81d-41f6-bf54-d1dec1d09f04 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:57.811742 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee80d9-f81d-41f6-bf54-d1dec1d09f04 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:57.812118 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee80d9-f81d-41f6-bf54-d1dec1d09f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:57.831552 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-1aee80d9-f81d-41f6-bf54-d1dec1d09f04 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:57.832774 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-1aee80d9-f81d-41f6-bf54-d1dec1d09f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:57.841302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1387/2753] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:57.846251 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67debbd-dd9f-41c6-ab00-416f651f49b5 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:57.859907 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67debbd-dd9f-41c6-ab00-416f651f49b5 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:57.865174 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abcbf5-0169-4a90-a9f4-001d52d294e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:57.865821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abcbf5-0169-4a90-a9f4-001d52d294e9 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:57.866098 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abcbf5-0169-4a90-a9f4-001d52d294e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:57.866362 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abcbf5-0169-4a90-a9f4-001d52d294e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:57.867080 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8abcbf5-0169-4a90-a9f4-001d52d294e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 10:26:57.867377 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8abcbf5-0169-4a90-a9f4-001d52d294e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:57.869008 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8abcbf5-0169-4a90-a9f4-001d52d294e9 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:57.883867 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1367/2754] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 10:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 10:26:57.890124 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee80d9-f81d-41f6-bf54-d1dec1d09f04 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:57.904918 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee80d9-f81d-41f6-bf54-d1dec1d09f04 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:57.913621 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec7fc25-350e-48a6-8fd4-96f559e25e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:57.914536 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7fc25-350e-48a6-8fd4-96f559e25e4b admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:57.914880 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7fc25-350e-48a6-8fd4-96f559e25e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:57.915170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7fc25-350e-48a6-8fd4-96f559e25e4b admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:57.916253 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec7fc25-350e-48a6-8fd4-96f559e25e4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:57.919120 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec7fc25-350e-48a6-8fd4-96f559e25e4b admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:57.950264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1388/2755] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:26:57 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:26:57.954721 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abcbf5-0169-4a90-a9f4-001d52d294e9 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:57.965469 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abcbf5-0169-4a90-a9f4-001d52d294e9 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:58.027825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abca147-2c54-40b7-ab10-cee51fea51e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:58.028617 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abca147-2c54-40b7-ab10-cee51fea51e3 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:58.028940 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abca147-2c54-40b7-ab10-cee51fea51e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:58.029208 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abca147-2c54-40b7-ab10-cee51fea51e3 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:58.029941 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abca147-2c54-40b7-ab10-cee51fea51e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:58.031920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abca147-2c54-40b7-ab10-cee51fea51e3 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:58.054042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1368/2756] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:26:57 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:26:58.059241 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec7fc25-350e-48a6-8fd4-96f559e25e4b admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:58.069421 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec7fc25-350e-48a6-8fd4-96f559e25e4b admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:58.074386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98363a8-c16a-48dc-9d23-45a7306863fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:58.075062 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98363a8-c16a-48dc-9d23-45a7306863fe admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:58.075351 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98363a8-c16a-48dc-9d23-45a7306863fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:58.075623 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98363a8-c16a-48dc-9d23-45a7306863fe admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:58.076400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98363a8-c16a-48dc-9d23-45a7306863fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:58.078034 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98363a8-c16a-48dc-9d23-45a7306863fe admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:58.088956 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1389/2757] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:58.093477 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abca147-2c54-40b7-ab10-cee51fea51e3 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:58.105015 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abca147-2c54-40b7-ab10-cee51fea51e3 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:58.113339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595adf69-85f7-4948-b9c2-2f327952c683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:58.114094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595adf69-85f7-4948-b9c2-2f327952c683 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:58.114372 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595adf69-85f7-4948-b9c2-2f327952c683 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:58.114713 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595adf69-85f7-4948-b9c2-2f327952c683 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:58.115607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595adf69-85f7-4948-b9c2-2f327952c683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:58.117545 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595adf69-85f7-4948-b9c2-2f327952c683 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:58.130509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1369/2758] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:58.137178 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98363a8-c16a-48dc-9d23-45a7306863fe admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:58.152024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98363a8-c16a-48dc-9d23-45a7306863fe admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:58.158056 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f9b4d-aa17-40eb-ae51-32784f88177e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:58.158938 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f9b4d-aa17-40eb-ae51-32784f88177e admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:58.159589 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f9b4d-aa17-40eb-ae51-32784f88177e admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:58.159863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f9b4d-aa17-40eb-ae51-32784f88177e admin admin] PATH_INFO: `/v3/projects/fc865f195ec24ac4896168078d4c8c48/users/5d55c17031f6409bad6913ecebc59942/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:58.174852 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f9b4d-aa17-40eb-ae51-32784f88177e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc865f195ec24ac4896168078d4c8c48, user_id=5d55c17031f6409bad6913ecebc59942, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:58.177834 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f9b4d-aa17-40eb-ae51-32784f88177e admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:58.193063 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1390/2759] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:26:58 2022] PUT /identity/v3/projects/fc865f195ec24ac4896168078d4c8c48/users/5d55c17031f6409bad6913ecebc59942/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:58.208412 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a76a8-8492-4d78-9500-f01a8c4477ca None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:58.208738 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a76a8-8492-4d78-9500-f01a8c4477ca None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:58.209031 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a76a8-8492-4d78-9500-f01a8c4477ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:58.236992 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-2a0a76a8-8492-4d78-9500-f01a8c4477ca None None] MFA Rules not processed for user `5d55c17031f6409bad6913ecebc59942`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:58.238191 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-2a0a76a8-8492-4d78-9500-f01a8c4477ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:58.305079 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1370/2760] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:58.309589 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f9b4d-aa17-40eb-ae51-32784f88177e admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:58.375171 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f9b4d-aa17-40eb-ae51-32784f88177e admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:58.380477 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37636fe-96e1-498e-8a3d-4852d75a0fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:58.381127 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37636fe-96e1-498e-8a3d-4852d75a0fac admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:58.381384 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37636fe-96e1-498e-8a3d-4852d75a0fac admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:58.381684 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37636fe-96e1-498e-8a3d-4852d75a0fac admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:58.382356 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37636fe-96e1-498e-8a3d-4852d75a0fac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:58.384287 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37636fe-96e1-498e-8a3d-4852d75a0fac admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:58.410936 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1391/2761] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:26:58 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 10:26:58.415132 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a76a8-8492-4d78-9500-f01a8c4477ca None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:58.426354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a76a8-8492-4d78-9500-f01a8c4477ca None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:58.491961 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee3e42-c510-4bcf-a754-5e353a2d08ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:58.492633 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee3e42-c510-4bcf-a754-5e353a2d08ad admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:58.492931 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee3e42-c510-4bcf-a754-5e353a2d08ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:58.493221 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee3e42-c510-4bcf-a754-5e353a2d08ad admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:58.493907 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee3e42-c510-4bcf-a754-5e353a2d08ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:58.496130 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee3e42-c510-4bcf-a754-5e353a2d08ad admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:58.524527 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1371/2762] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:26:58 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:26:58.528755 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37636fe-96e1-498e-8a3d-4852d75a0fac admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:58.539826 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37636fe-96e1-498e-8a3d-4852d75a0fac admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:58.545322 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9738f-35fe-42a3-a7e2-df70572d4170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:58.545983 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9738f-35fe-42a3-a7e2-df70572d4170 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:58.546264 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9738f-35fe-42a3-a7e2-df70572d4170 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:58.547166 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9738f-35fe-42a3-a7e2-df70572d4170 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:58.547913 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9738f-35fe-42a3-a7e2-df70572d4170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:58.549562 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9738f-35fe-42a3-a7e2-df70572d4170 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:58.560784 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1392/2763] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:58.565012 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee3e42-c510-4bcf-a754-5e353a2d08ad admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:58.575354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee3e42-c510-4bcf-a754-5e353a2d08ad admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:58.580509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c8b5c-5bf3-4a2b-89db-10336d5fcbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:58.581525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c8b5c-5bf3-4a2b-89db-10336d5fcbc7 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:58.581803 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c8b5c-5bf3-4a2b-89db-10336d5fcbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:58.582072 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c8b5c-5bf3-4a2b-89db-10336d5fcbc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:58.582799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c8b5c-5bf3-4a2b-89db-10336d5fcbc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:58.584460 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c8b5c-5bf3-4a2b-89db-10336d5fcbc7 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:58.595059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1372/2764] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:58.599396 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9738f-35fe-42a3-a7e2-df70572d4170 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:58.609503 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9738f-35fe-42a3-a7e2-df70572d4170 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:58.614522 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7136e-b77f-4892-a5f3-397a9261b3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:58.615421 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7136e-b77f-4892-a5f3-397a9261b3fd admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:58.615684 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7136e-b77f-4892-a5f3-397a9261b3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:58.615952 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7136e-b77f-4892-a5f3-397a9261b3fd admin admin] PATH_INFO: `/v3/projects/9dd83034996b43babdf63402ef8fc4a7/users/e6f2cd65eca145b49773de24614f5810/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:58.630886 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7136e-b77f-4892-a5f3-397a9261b3fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dd83034996b43babdf63402ef8fc4a7, user_id=e6f2cd65eca145b49773de24614f5810, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:58.633708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7136e-b77f-4892-a5f3-397a9261b3fd admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:58.647659 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1393/2765] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:26:58 2022] PUT /identity/v3/projects/9dd83034996b43babdf63402ef8fc4a7/users/e6f2cd65eca145b49773de24614f5810/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 10:26:58.660082 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bf28d-2941-458e-ab08-afd7c614dba3 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:58.660575 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bf28d-2941-458e-ab08-afd7c614dba3 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:58.661039 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bf28d-2941-458e-ab08-afd7c614dba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:58.691111 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-a77bf28d-2941-458e-ab08-afd7c614dba3 None None] MFA Rules not processed for user `e6f2cd65eca145b49773de24614f5810`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:58.692320 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a77bf28d-2941-458e-ab08-afd7c614dba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:58.769599 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1373/2766] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:58.775876 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7136e-b77f-4892-a5f3-397a9261b3fd admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:58.844812 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7136e-b77f-4892-a5f3-397a9261b3fd admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:58.850498 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a8dc8-3534-4d87-9006-3dd4b3260055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:58.851167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a8dc8-3534-4d87-9006-3dd4b3260055 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:58.851443 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a8dc8-3534-4d87-9006-3dd4b3260055 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:58.851708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a8dc8-3534-4d87-9006-3dd4b3260055 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:58.852402 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a8dc8-3534-4d87-9006-3dd4b3260055 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:58.854352 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a8dc8-3534-4d87-9006-3dd4b3260055 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:58.881499 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1394/2767] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:26:58 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:26:58.885145 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77bf28d-2941-458e-ab08-afd7c614dba3 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:58.903981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77bf28d-2941-458e-ab08-afd7c614dba3 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:58.965467 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac33ee-ac55-472a-8a26-fdba5d6355b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:58.966168 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac33ee-ac55-472a-8a26-fdba5d6355b3 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:58.966454 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac33ee-ac55-472a-8a26-fdba5d6355b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:58.966771 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac33ee-ac55-472a-8a26-fdba5d6355b3 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:58.967538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac33ee-ac55-472a-8a26-fdba5d6355b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:58.969537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac33ee-ac55-472a-8a26-fdba5d6355b3 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:58.991122 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1374/2768] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:26:58 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 10:26:58.995140 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a8dc8-3534-4d87-9006-3dd4b3260055 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:59.006112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a8dc8-3534-4d87-9006-3dd4b3260055 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:59.011076 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527dda1-836c-4d4a-a904-48751b2b865a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:59.011741 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527dda1-836c-4d4a-a904-48751b2b865a admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:59.012182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527dda1-836c-4d4a-a904-48751b2b865a admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:59.012293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527dda1-836c-4d4a-a904-48751b2b865a admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:59.013046 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527dda1-836c-4d4a-a904-48751b2b865a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:59.014744 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527dda1-836c-4d4a-a904-48751b2b865a admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:59.029177 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1395/2769] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:59.030597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac33ee-ac55-472a-8a26-fdba5d6355b3 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:59.041265 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac33ee-ac55-472a-8a26-fdba5d6355b3 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:59.047094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da37d3-5f81-4e32-ad01-b3a30198ed73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:59.048011 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da37d3-5f81-4e32-ad01-b3a30198ed73 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:59.048262 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da37d3-5f81-4e32-ad01-b3a30198ed73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:59.048525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da37d3-5f81-4e32-ad01-b3a30198ed73 admin admin] PATH_INFO: `/v3/projects/338ca84f429c4d31b5dab831c7016e46/users/d8c9907830624c439049525f4795f9c4/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:59.064324 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da37d3-5f81-4e32-ad01-b3a30198ed73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=338ca84f429c4d31b5dab831c7016e46, user_id=d8c9907830624c439049525f4795f9c4, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:59.067082 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da37d3-5f81-4e32-ad01-b3a30198ed73 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:59.082460 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1375/2770] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:26:59 2022] PUT /identity/v3/projects/338ca84f429c4d31b5dab831c7016e46/users/d8c9907830624c439049525f4795f9c4/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:59.086369 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527dda1-836c-4d4a-a904-48751b2b865a admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:59.096970 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527dda1-836c-4d4a-a904-48751b2b865a admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:59.154571 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09712bc-24ae-4525-9ce9-6d265bf42398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:59.155244 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09712bc-24ae-4525-9ce9-6d265bf42398 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:59.155527 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09712bc-24ae-4525-9ce9-6d265bf42398 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:59.155795 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09712bc-24ae-4525-9ce9-6d265bf42398 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:59.156511 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09712bc-24ae-4525-9ce9-6d265bf42398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:59.158145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09712bc-24ae-4525-9ce9-6d265bf42398 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:59.167794 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1396/2771] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:26:59.172003 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da37d3-5f81-4e32-ad01-b3a30198ed73 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:59.182520 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da37d3-5f81-4e32-ad01-b3a30198ed73 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:59.187315 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b68be5-030d-4a4b-b671-c32b9cbb9265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:59.188198 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b68be5-030d-4a4b-b671-c32b9cbb9265 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:59.188475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b68be5-030d-4a4b-b671-c32b9cbb9265 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:59.188770 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b68be5-030d-4a4b-b671-c32b9cbb9265 admin admin] PATH_INFO: `/v3/projects/338ca84f429c4d31b5dab831c7016e46/users/d8c9907830624c439049525f4795f9c4/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:59.190854 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b68be5-030d-4a4b-b671-c32b9cbb9265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=338ca84f429c4d31b5dab831c7016e46, user_id=d8c9907830624c439049525f4795f9c4, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:59.193595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b68be5-030d-4a4b-b671-c32b9cbb9265 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:59.204779 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1376/2772] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:26:59 2022] PUT /identity/v3/projects/338ca84f429c4d31b5dab831c7016e46/users/d8c9907830624c439049525f4795f9c4/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:26:59.215501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f784d-2bb5-4519-80a7-9cc9c5e88f89 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:59.215858 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f784d-2bb5-4519-80a7-9cc9c5e88f89 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:59.216195 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f784d-2bb5-4519-80a7-9cc9c5e88f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:59.242889 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-192f784d-2bb5-4519-80a7-9cc9c5e88f89 None None] MFA Rules not processed for user `d8c9907830624c439049525f4795f9c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:26:59.244194 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-192f784d-2bb5-4519-80a7-9cc9c5e88f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:59.313613 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1397/2773] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:26:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:26:59.325171 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b68be5-030d-4a4b-b671-c32b9cbb9265 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:26:59.327260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-14b68be5-030d-4a4b-b671-c32b9cbb9265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:26:59.387864 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b68be5-030d-4a4b-b671-c32b9cbb9265 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:26:59.391744 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b072fee-a00b-4c7e-a23e-c82204d97a33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:26:59.392474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b072fee-a00b-4c7e-a23e-c82204d97a33 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:26:59.392859 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b072fee-a00b-4c7e-a23e-c82204d97a33 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:26:59.393131 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b072fee-a00b-4c7e-a23e-c82204d97a33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:26:59.454880 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b072fee-a00b-4c7e-a23e-c82204d97a33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:26:59.456688 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b072fee-a00b-4c7e-a23e-c82204d97a33 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:26:59.467064 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1377/2774] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:26:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 10:27:00.968693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f784d-2bb5-4519-80a7-9cc9c5e88f89 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:27:00.970131 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-192f784d-2bb5-4519-80a7-9cc9c5e88f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:27:00.984185 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f784d-2bb5-4519-80a7-9cc9c5e88f89 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:27:00.988299 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d8a71-f180-4792-b64c-ed0dfbce8729 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:27:00.989059 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d8a71-f180-4792-b64c-ed0dfbce8729 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:00.989275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d8a71-f180-4792-b64c-ed0dfbce8729 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:00.989546 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d8a71-f180-4792-b64c-ed0dfbce8729 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:01.064050 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d8a71-f180-4792-b64c-ed0dfbce8729 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:27:01.065645 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d8a71-f180-4792-b64c-ed0dfbce8729 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:27:01.075943 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1398/2775] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:27:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:27:01.666869 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b072fee-a00b-4c7e-a23e-c82204d97a33 service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:27:01.684445 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b072fee-a00b-4c7e-a23e-c82204d97a33 service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:27:01.690684 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741701f-b1c6-46fe-a869-03ff85876cf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:27:01.695305 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741701f-b1c6-46fe-a869-03ff85876cf0 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:01.695977 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741701f-b1c6-46fe-a869-03ff85876cf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:01.696256 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741701f-b1c6-46fe-a869-03ff85876cf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:01.709952 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741701f-b1c6-46fe-a869-03ff85876cf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:27:01.711920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741701f-b1c6-46fe-a869-03ff85876cf0 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:27:01.732752 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1378/2776] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:27:01 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:27:05.636806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d8a71-f180-4792-b64c-ed0dfbce8729 service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:27:05.700000 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d8a71-f180-4792-b64c-ed0dfbce8729 service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:27:05.708209 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a011651-f515-43d8-9935-6a7e0ea7a19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:27:05.709554 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a011651-f515-43d8-9935-6a7e0ea7a19b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:05.710024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a011651-f515-43d8-9935-6a7e0ea7a19b admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:05.710454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a011651-f515-43d8-9935-6a7e0ea7a19b admin admin] PATH_INFO: `/v3/users/5d55c17031f6409bad6913ecebc59942` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:05.712494 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a011651-f515-43d8-9935-6a7e0ea7a19b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d55c17031f6409bad6913ecebc59942)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:27:05.715656 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a011651-f515-43d8-9935-6a7e0ea7a19b admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:27:05.774640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-2a011651-f515-43d8-9935-6a7e0ea7a19b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d55c17031f6409bad6913ecebc59942'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:05.784835 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-2a011651-f515-43d8-9935-6a7e0ea7a19b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d55c17031f6409bad6913ecebc59942'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:05.785598 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-2a011651-f515-43d8-9935-6a7e0ea7a19b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d55c17031f6409bad6913ecebc59942'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:05.795008 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-2a011651-f515-43d8-9935-6a7e0ea7a19b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d55c17031f6409bad6913ecebc59942'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:05.795627 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-2a011651-f515-43d8-9935-6a7e0ea7a19b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d55c17031f6409bad6913ecebc59942'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:05.815795 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1399/2777] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:27:05 2022] DELETE /identity/v3/users/5d55c17031f6409bad6913ecebc59942 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:27:05.820173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa362e-a3f3-4623-aba3-c8830f20294c None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:05.820691 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa362e-a3f3-4623-aba3-c8830f20294c None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:05.820846 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa362e-a3f3-4623-aba3-c8830f20294c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:05.838977 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-12fa362e-a3f3-4623-aba3-c8830f20294c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:27:05.840171 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-12fa362e-a3f3-4623-aba3-c8830f20294c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:27:05.900509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1379/2778] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:27:05.912035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a011651-f515-43d8-9935-6a7e0ea7a19b admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:27:05.913595 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-2a011651-f515-43d8-9935-6a7e0ea7a19b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:27:05.974084 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a011651-f515-43d8-9935-6a7e0ea7a19b admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:27:05.978854 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1863e-7bd3-422b-9428-ce1a9a02d08f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:27:05.979511 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1863e-7bd3-422b-9428-ce1a9a02d08f service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:05.979777 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1863e-7bd3-422b-9428-ce1a9a02d08f service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:05.980048 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1863e-7bd3-422b-9428-ce1a9a02d08f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:05.993403 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1863e-7bd3-422b-9428-ce1a9a02d08f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:27:05.994861 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1863e-7bd3-422b-9428-ce1a9a02d08f service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:27:06.004822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1400/2779] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:27:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:27:06.286028 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa362e-a3f3-4623-aba3-c8830f20294c None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:27:06.287500 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-12fa362e-a3f3-4623-aba3-c8830f20294c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:27:06.300015 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa362e-a3f3-4623-aba3-c8830f20294c None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:27:06.305236 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b9b70-2bfb-45c5-9150-183643317e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:27:06.306124 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b9b70-2bfb-45c5-9150-183643317e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:06.306389 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b9b70-2bfb-45c5-9150-183643317e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:06.306656 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b9b70-2bfb-45c5-9150-183643317e2b admin admin] PATH_INFO: `/v3/projects/fc865f195ec24ac4896168078d4c8c48` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:06.307946 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b9b70-2bfb-45c5-9150-183643317e2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc865f195ec24ac4896168078d4c8c48)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:27:06.309931 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b9b70-2bfb-45c5-9150-183643317e2b admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:27:06.364032 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-db9b9b70-2bfb-45c5-9150-183643317e2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc865f195ec24ac4896168078d4c8c48', 'request_id': 'req-db9b9b70-2bfb-45c5-9150-183643317e2b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:06.370108 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-db9b9b70-2bfb-45c5-9150-183643317e2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc865f195ec24ac4896168078d4c8c48', 'request_id': 'req-db9b9b70-2bfb-45c5-9150-183643317e2b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:06.375516 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-db9b9b70-2bfb-45c5-9150-183643317e2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc865f195ec24ac4896168078d4c8c48', 'request_id': 'req-db9b9b70-2bfb-45c5-9150-183643317e2b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:06.390570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-db9b9b70-2bfb-45c5-9150-183643317e2b admin admin] The token cache is being invalidate because project fc865f195ec24ac4896168078d4c8c48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:27:06.390956 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-db9b9b70-2bfb-45c5-9150-183643317e2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:06.392061 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-db9b9b70-2bfb-45c5-9150-183643317e2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:06.394561 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1380/2780] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:27:06 2022] DELETE /identity/v3/projects/fc865f195ec24ac4896168078d4c8c48 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:27:06.398764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1863e-7bd3-422b-9428-ce1a9a02d08f service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:27:06.408979 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1863e-7bd3-422b-9428-ce1a9a02d08f service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:27:06.469141 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c4f04-3a84-44ab-99b2-0aea83cc3c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:27:06.470009 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c4f04-3a84-44ab-99b2-0aea83cc3c20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:06.470291 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c4f04-3a84-44ab-99b2-0aea83cc3c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:06.470559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c4f04-3a84-44ab-99b2-0aea83cc3c20 admin admin] PATH_INFO: `/v3/users/e6f2cd65eca145b49773de24614f5810` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:06.471793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c4f04-3a84-44ab-99b2-0aea83cc3c20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6f2cd65eca145b49773de24614f5810)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:27:06.473766 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c4f04-3a84-44ab-99b2-0aea83cc3c20 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:27:06.518744 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-190c4f04-3a84-44ab-99b2-0aea83cc3c20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6f2cd65eca145b49773de24614f5810'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:06.525194 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-190c4f04-3a84-44ab-99b2-0aea83cc3c20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6f2cd65eca145b49773de24614f5810'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:06.525841 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-190c4f04-3a84-44ab-99b2-0aea83cc3c20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6f2cd65eca145b49773de24614f5810'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:06.534191 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-190c4f04-3a84-44ab-99b2-0aea83cc3c20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6f2cd65eca145b49773de24614f5810'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:06.534798 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-190c4f04-3a84-44ab-99b2-0aea83cc3c20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6f2cd65eca145b49773de24614f5810'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:06.552087 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1401/2781] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:27:06 2022] DELETE /identity/v3/users/e6f2cd65eca145b49773de24614f5810 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:27:06.840042 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b9b70-2bfb-45c5-9150-183643317e2b admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:27:06.841604 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-db9b9b70-2bfb-45c5-9150-183643317e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:27:06.860385 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b9b70-2bfb-45c5-9150-183643317e2b admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:27:06.916635 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474b8a5-7aaa-452c-852c-ca9a35af79a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:27:06.917528 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474b8a5-7aaa-452c-852c-ca9a35af79a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:06.917802 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474b8a5-7aaa-452c-852c-ca9a35af79a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:06.918067 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474b8a5-7aaa-452c-852c-ca9a35af79a5 admin admin] PATH_INFO: `/v3/projects/9dd83034996b43babdf63402ef8fc4a7` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:06.919235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474b8a5-7aaa-452c-852c-ca9a35af79a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dd83034996b43babdf63402ef8fc4a7)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:27:06.921269 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474b8a5-7aaa-452c-852c-ca9a35af79a5 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:27:06.970360 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4474b8a5-7aaa-452c-852c-ca9a35af79a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dd83034996b43babdf63402ef8fc4a7', 'request_id': 'req-4474b8a5-7aaa-452c-852c-ca9a35af79a5'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:06.976833 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4474b8a5-7aaa-452c-852c-ca9a35af79a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dd83034996b43babdf63402ef8fc4a7', 'request_id': 'req-4474b8a5-7aaa-452c-852c-ca9a35af79a5'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:06.982800 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4474b8a5-7aaa-452c-852c-ca9a35af79a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dd83034996b43babdf63402ef8fc4a7', 'request_id': 'req-4474b8a5-7aaa-452c-852c-ca9a35af79a5'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:06.996520 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4474b8a5-7aaa-452c-852c-ca9a35af79a5 admin admin] The token cache is being invalidate because project 9dd83034996b43babdf63402ef8fc4a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:27:06.996841 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4474b8a5-7aaa-452c-852c-ca9a35af79a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:06.997489 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-4474b8a5-7aaa-452c-852c-ca9a35af79a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:06.999111 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1381/2782] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:27:06 2022] DELETE /identity/v3/projects/9dd83034996b43babdf63402ef8fc4a7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:27:07.003139 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c4f04-3a84-44ab-99b2-0aea83cc3c20 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:27:07.004610 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-190c4f04-3a84-44ab-99b2-0aea83cc3c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:27:07.017978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c4f04-3a84-44ab-99b2-0aea83cc3c20 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:27:07.076759 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b98bd7a-96f6-4955-a167-cdafe3b5ca98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:27:07.077651 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98bd7a-96f6-4955-a167-cdafe3b5ca98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:07.077924 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98bd7a-96f6-4955-a167-cdafe3b5ca98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:07.078195 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98bd7a-96f6-4955-a167-cdafe3b5ca98 admin admin] PATH_INFO: `/v3/users/d8c9907830624c439049525f4795f9c4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:07.079403 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b98bd7a-96f6-4955-a167-cdafe3b5ca98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8c9907830624c439049525f4795f9c4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:27:07.081698 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b98bd7a-96f6-4955-a167-cdafe3b5ca98 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:27:07.131604 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-7b98bd7a-96f6-4955-a167-cdafe3b5ca98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8c9907830624c439049525f4795f9c4'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:07.138489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-7b98bd7a-96f6-4955-a167-cdafe3b5ca98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8c9907830624c439049525f4795f9c4'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:07.139155 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-7b98bd7a-96f6-4955-a167-cdafe3b5ca98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8c9907830624c439049525f4795f9c4'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:07.147359 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-7b98bd7a-96f6-4955-a167-cdafe3b5ca98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8c9907830624c439049525f4795f9c4'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:07.147978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-7b98bd7a-96f6-4955-a167-cdafe3b5ca98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8c9907830624c439049525f4795f9c4'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:07.164539 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1402/2783] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:27:07 2022] DELETE /identity/v3/users/d8c9907830624c439049525f4795f9c4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:27:07.440349 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474b8a5-7aaa-452c-852c-ca9a35af79a5 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:27:07.442166 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-4474b8a5-7aaa-452c-852c-ca9a35af79a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:27:07.456799 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474b8a5-7aaa-452c-852c-ca9a35af79a5 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:27:07.524345 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c19ce2-1bcd-48fa-be8e-ad5d0e23ae72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:27:07.525242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c19ce2-1bcd-48fa-be8e-ad5d0e23ae72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:07.525509 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c19ce2-1bcd-48fa-be8e-ad5d0e23ae72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:07.525775 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c19ce2-1bcd-48fa-be8e-ad5d0e23ae72 admin admin] PATH_INFO: `/v3/projects/338ca84f429c4d31b5dab831c7016e46` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:07.527084 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c19ce2-1bcd-48fa-be8e-ad5d0e23ae72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=338ca84f429c4d31b5dab831c7016e46)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:27:07.529098 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c19ce2-1bcd-48fa-be8e-ad5d0e23ae72 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:27:07.581429 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-02c19ce2-1bcd-48fa-be8e-ad5d0e23ae72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '338ca84f429c4d31b5dab831c7016e46', 'request_id': 'req-02c19ce2-1bcd-48fa-be8e-ad5d0e23ae72'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:07.587741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-02c19ce2-1bcd-48fa-be8e-ad5d0e23ae72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '338ca84f429c4d31b5dab831c7016e46', 'request_id': 'req-02c19ce2-1bcd-48fa-be8e-ad5d0e23ae72'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:07.593607 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-02c19ce2-1bcd-48fa-be8e-ad5d0e23ae72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '338ca84f429c4d31b5dab831c7016e46', 'request_id': 'req-02c19ce2-1bcd-48fa-be8e-ad5d0e23ae72'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:07.609909 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-02c19ce2-1bcd-48fa-be8e-ad5d0e23ae72 admin admin] The token cache is being invalidate because project 338ca84f429c4d31b5dab831c7016e46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:27:07.610193 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-02c19ce2-1bcd-48fa-be8e-ad5d0e23ae72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:07.610825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-02c19ce2-1bcd-48fa-be8e-ad5d0e23ae72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:27:07.612395 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1382/2784] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:27:07 2022] DELETE /identity/v3/projects/338ca84f429c4d31b5dab831c7016e46 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:27:07.617815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f210efc-2fd3-4ef2-8a2a-8477e7c780b9 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:07.618096 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f210efc-2fd3-4ef2-8a2a-8477e7c780b9 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:07.618369 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f210efc-2fd3-4ef2-8a2a-8477e7c780b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:07.636857 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-6f210efc-2fd3-4ef2-8a2a-8477e7c780b9 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:27:07.638081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-6f210efc-2fd3-4ef2-8a2a-8477e7c780b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:27:07.696124 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1403/2785] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:27:07.704049 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c9bcc-af6c-40d2-9a02-1c6001fc4f81 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:07.704333 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c9bcc-af6c-40d2-9a02-1c6001fc4f81 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:07.704605 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c9bcc-af6c-40d2-9a02-1c6001fc4f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:07.724621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-b17c9bcc-af6c-40d2-9a02-1c6001fc4f81 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:27:07.725846 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-b17c9bcc-af6c-40d2-9a02-1c6001fc4f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:27:07.734562 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1383/2786] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:27:07.738983 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f210efc-2fd3-4ef2-8a2a-8477e7c780b9 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:27:07.748653 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f210efc-2fd3-4ef2-8a2a-8477e7c780b9 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:27:07.753860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b281d84-c0f4-4a6b-9443-b99976d971bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:27:07.754508 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b281d84-c0f4-4a6b-9443-b99976d971bd admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:07.754783 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b281d84-c0f4-4a6b-9443-b99976d971bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:07.755050 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b281d84-c0f4-4a6b-9443-b99976d971bd admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:07.755784 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b281d84-c0f4-4a6b-9443-b99976d971bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 10:27:07.756079 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b281d84-c0f4-4a6b-9443-b99976d971bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:27:07.757723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b281d84-c0f4-4a6b-9443-b99976d971bd admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:27:07.770517 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1404/2787] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 10:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 10:27:07.774936 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c9bcc-af6c-40d2-9a02-1c6001fc4f81 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:27:07.787549 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c9bcc-af6c-40d2-9a02-1c6001fc4f81 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:27:07.793623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f8504-3c23-4005-be5a-233777bda9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:27:07.794277 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f8504-3c23-4005-be5a-233777bda9d7 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:07.794554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f8504-3c23-4005-be5a-233777bda9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:07.794952 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f8504-3c23-4005-be5a-233777bda9d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:07.795631 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f8504-3c23-4005-be5a-233777bda9d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:27:07.797691 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f8504-3c23-4005-be5a-233777bda9d7 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:27:07.832870 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1384/2788] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:27:07 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:27:07.836042 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b281d84-c0f4-4a6b-9443-b99976d971bd admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:27:07.846182 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b281d84-c0f4-4a6b-9443-b99976d971bd admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:27:07.914241 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f211a0c-13b1-4aa6-8ad8-b9c58c77447a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:27:07.914901 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f211a0c-13b1-4aa6-8ad8-b9c58c77447a admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:07.915222 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f211a0c-13b1-4aa6-8ad8-b9c58c77447a admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:07.915616 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f211a0c-13b1-4aa6-8ad8-b9c58c77447a admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:07.916492 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f211a0c-13b1-4aa6-8ad8-b9c58c77447a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:27:07.918319 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f211a0c-13b1-4aa6-8ad8-b9c58c77447a admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:27:07.939400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1405/2789] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:27:07 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 10:27:07.943589 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f8504-3c23-4005-be5a-233777bda9d7 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:27:07.954941 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f8504-3c23-4005-be5a-233777bda9d7 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:27:07.960691 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5145b57-d6d9-470b-804c-6d852fda6921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:27:07.961358 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5145b57-d6d9-470b-804c-6d852fda6921 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:07.961636 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5145b57-d6d9-470b-804c-6d852fda6921 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:07.961911 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5145b57-d6d9-470b-804c-6d852fda6921 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:07.962681 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5145b57-d6d9-470b-804c-6d852fda6921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:27:07.964289 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5145b57-d6d9-470b-804c-6d852fda6921 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:27:07.976314 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1385/2790] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:27:07.980759 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f211a0c-13b1-4aa6-8ad8-b9c58c77447a admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:27:07.991952 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f211a0c-13b1-4aa6-8ad8-b9c58c77447a admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:27:07.998525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725072aa-f19d-455c-ba9b-2370f86d67c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:27:07.999181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725072aa-f19d-455c-ba9b-2370f86d67c7 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:07.999451 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725072aa-f19d-455c-ba9b-2370f86d67c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:07.999709 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725072aa-f19d-455c-ba9b-2370f86d67c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:08.000435 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725072aa-f19d-455c-ba9b-2370f86d67c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:27:08.002075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725072aa-f19d-455c-ba9b-2370f86d67c7 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:27:08.012644 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1406/2791] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:27:08.018114 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5145b57-d6d9-470b-804c-6d852fda6921 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:27:08.034904 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5145b57-d6d9-470b-804c-6d852fda6921 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:27:08.040648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f42d64-d05e-4f9e-99f1-7ad7b486bccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:27:08.041768 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f42d64-d05e-4f9e-99f1-7ad7b486bccc admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:08.042046 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f42d64-d05e-4f9e-99f1-7ad7b486bccc admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:08.042352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f42d64-d05e-4f9e-99f1-7ad7b486bccc admin admin] PATH_INFO: `/v3/projects/438ea2009cc142578a97d17a081e8862/users/362a6e9c40944d8e8d818a1300c81983/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:08.058968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f42d64-d05e-4f9e-99f1-7ad7b486bccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=438ea2009cc142578a97d17a081e8862, user_id=362a6e9c40944d8e8d818a1300c81983, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:27:08.061874 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f42d64-d05e-4f9e-99f1-7ad7b486bccc admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:27:08.077754 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1386/2792] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:27:08 2022] PUT /identity/v3/projects/438ea2009cc142578a97d17a081e8862/users/362a6e9c40944d8e8d818a1300c81983/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:27:08.083332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34bbee-2548-40ba-b879-72c185ed215a None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:08.083639 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34bbee-2548-40ba-b879-72c185ed215a None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:08.083919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34bbee-2548-40ba-b879-72c185ed215a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:08.102989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-bd34bbee-2548-40ba-b879-72c185ed215a None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:27:08.104316 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-bd34bbee-2548-40ba-b879-72c185ed215a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:27:08.163860 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1407/2793] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:27:08.173688 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f42d64-d05e-4f9e-99f1-7ad7b486bccc admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:27:08.175665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a7f42d64-d05e-4f9e-99f1-7ad7b486bccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:27:08.233600 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f42d64-d05e-4f9e-99f1-7ad7b486bccc admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:27:08.237420 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38712d3-66b8-47fb-b3eb-118c8802deba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:27:08.238066 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38712d3-66b8-47fb-b3eb-118c8802deba service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:08.238339 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38712d3-66b8-47fb-b3eb-118c8802deba service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:08.238603 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38712d3-66b8-47fb-b3eb-118c8802deba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:08.249864 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38712d3-66b8-47fb-b3eb-118c8802deba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:27:08.251320 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38712d3-66b8-47fb-b3eb-118c8802deba service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:27:08.260933 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1387/2794] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:27:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:27:08.882602 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51641f9b-9969-4750-9277-5835fe300195 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:08.883036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51641f9b-9969-4750-9277-5835fe300195 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:08.883454 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51641f9b-9969-4750-9277-5835fe300195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:08.917167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-51641f9b-9969-4750-9277-5835fe300195 None None] MFA Rules not processed for user `362a6e9c40944d8e8d818a1300c81983`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:27:08.917167 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-51641f9b-9969-4750-9277-5835fe300195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:27:08.982626 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1408/2795] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:27:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:27:08.993437 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38712d3-66b8-47fb-b3eb-118c8802deba service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:27:08.994850 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f38712d3-66b8-47fb-b3eb-118c8802deba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:27:09.053398 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38712d3-66b8-47fb-b3eb-118c8802deba service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:27:09.057298 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a637b-7af2-4708-b035-143b704f71e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:27:09.057962 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a637b-7af2-4708-b035-143b704f71e6 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:09.058237 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a637b-7af2-4708-b035-143b704f71e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:09.058501 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a637b-7af2-4708-b035-143b704f71e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:09.067910 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a637b-7af2-4708-b035-143b704f71e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:27:09.069380 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a637b-7af2-4708-b035-143b704f71e6 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:27:09.078040 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1388/2796] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:27:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:27:10.448161 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030917b9-cf54-4783-bb59-63ff6e5e34ae None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:10.448731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030917b9-cf54-4783-bb59-63ff6e5e34ae None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:10.448731 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030917b9-cf54-4783-bb59-63ff6e5e34ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:10.467437 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-030917b9-cf54-4783-bb59-63ff6e5e34ae None None] MFA Rules not processed for user `362a6e9c40944d8e8d818a1300c81983`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:27:10.468656 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-030917b9-cf54-4783-bb59-63ff6e5e34ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:27:10.476730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1409/2797] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:27:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:27:17.920603 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a637b-7af2-4708-b035-143b704f71e6 service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:27:17.922126 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-157a637b-7af2-4708-b035-143b704f71e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:27:17.934035 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a637b-7af2-4708-b035-143b704f71e6 service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:27:17.937884 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2146462-530f-4a9b-8a28-7ef40b9beb62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:27:17.938537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2146462-530f-4a9b-8a28-7ef40b9beb62 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:17.938808 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2146462-530f-4a9b-8a28-7ef40b9beb62 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:17.939082 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2146462-530f-4a9b-8a28-7ef40b9beb62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:17.940894 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f2146462-530f-4a9b-8a28-7ef40b9beb62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:27:18.006389 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2146462-530f-4a9b-8a28-7ef40b9beb62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:27:18.008354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2146462-530f-4a9b-8a28-7ef40b9beb62 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:27:18.019774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1389/2798] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:27:17 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 10:27:28.379341 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa996ee-b2e4-4523-9cfb-d64b8a3b451b None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:28.379896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa996ee-b2e4-4523-9cfb-d64b8a3b451b None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:28.379896 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa996ee-b2e4-4523-9cfb-d64b8a3b451b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:28.398096 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-aaa996ee-b2e4-4523-9cfb-d64b8a3b451b None None] MFA Rules not processed for user `362a6e9c40944d8e8d818a1300c81983`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:27:28.399294 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-aaa996ee-b2e4-4523-9cfb-d64b8a3b451b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:27:28.407204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1410/2799] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:27:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:27:28.651001 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2146462-530f-4a9b-8a28-7ef40b9beb62 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:27:28.664336 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2146462-530f-4a9b-8a28-7ef40b9beb62 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:27:28.670479 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d60ef-7ca6-402c-bc2b-be868bc510e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:27:28.671161 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d60ef-7ca6-402c-bc2b-be868bc510e9 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:28.671464 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d60ef-7ca6-402c-bc2b-be868bc510e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:28.671731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d60ef-7ca6-402c-bc2b-be868bc510e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:28.673945 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-ad2d60ef-7ca6-402c-bc2b-be868bc510e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:27:28.684636 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d60ef-7ca6-402c-bc2b-be868bc510e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:27:28.686102 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d60ef-7ca6-402c-bc2b-be868bc510e9 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:27:28.695189 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1390/2800] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:27:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:27:52.177161 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda5fdb-8b5b-46dd-a4b3-018e2d371e8e None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:52.177654 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda5fdb-8b5b-46dd-a4b3-018e2d371e8e None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:52.177780 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda5fdb-8b5b-46dd-a4b3-018e2d371e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:52.195989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-5cda5fdb-8b5b-46dd-a4b3-018e2d371e8e None None] MFA Rules not processed for user `af62b2d3f73f482b8db4173af33752c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:27:52.197238 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-5cda5fdb-8b5b-46dd-a4b3-018e2d371e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:27:52.258143 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1411/2801] 10.222.0.26 () {58 vars in 1009 bytes} [Mon May 9 10:27:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:27:52.313541 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d60ef-7ca6-402c-bc2b-be868bc510e9 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:27:52.315010 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-ad2d60ef-7ca6-402c-bc2b-be868bc510e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:27:52.384825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d60ef-7ca6-402c-bc2b-be868bc510e9 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:27:52.390249 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e1d21-feaa-483e-8f30-102740f41093 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380e04358f2349aa912d6a97518877cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:27:52.390890 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e1d21-feaa-483e-8f30-102740f41093 service swift] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:27:52.391167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e1d21-feaa-483e-8f30-102740f41093 service swift] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:27:52.391437 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e1d21-feaa-483e-8f30-102740f41093 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:27:52.403870 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e1d21-feaa-483e-8f30-102740f41093 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:27:52.405734 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e1d21-feaa-483e-8f30-102740f41093 service swift] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:27:52.415252 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1391/2802] 10.222.0.26 () {60 vars in 1384 bytes} [Mon May 9 10:27:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 10:28:06.431315 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664c99a-fabc-42d8-9cae-575431302616 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:06.431822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664c99a-fabc-42d8-9cae-575431302616 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:06.431822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664c99a-fabc-42d8-9cae-575431302616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:06.450789 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-2664c99a-fabc-42d8-9cae-575431302616 None None] MFA Rules not processed for user `af62b2d3f73f482b8db4173af33752c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:28:06.451984 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-2664c99a-fabc-42d8-9cae-575431302616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:28:06.461985 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1412/2803] 10.222.0.26 () {58 vars in 967 bytes} [Mon May 9 10:28:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:28:06.476790 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e1d21-feaa-483e-8f30-102740f41093 service swift] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:28:06.492888 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e1d21-feaa-483e-8f30-102740f41093 service swift] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:28:06.499786 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddd4c6-8d6f-432d-80b9-c27d410fa458 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380e04358f2349aa912d6a97518877cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:28:06.500421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddd4c6-8d6f-432d-80b9-c27d410fa458 service swift] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:06.500720 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddd4c6-8d6f-432d-80b9-c27d410fa458 service swift] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:06.500974 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddd4c6-8d6f-432d-80b9-c27d410fa458 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:06.509795 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddd4c6-8d6f-432d-80b9-c27d410fa458 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:28:06.511373 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddd4c6-8d6f-432d-80b9-c27d410fa458 service swift] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:28:06.519512 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1392/2804] 10.222.0.26 () {60 vars in 1384 bytes} [Mon May 9 10:28:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 10:28:08.474283 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664c99a-fabc-42d8-9cae-575431302616 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:28:08.486278 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664c99a-fabc-42d8-9cae-575431302616 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:28:08.491325 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e39097-280e-425f-869a-fc0c01ae1c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:28:08.492216 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e39097-280e-425f-869a-fc0c01ae1c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:08.492489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e39097-280e-425f-869a-fc0c01ae1c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:08.492777 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e39097-280e-425f-869a-fc0c01ae1c6e admin admin] PATH_INFO: `/v3/users/362a6e9c40944d8e8d818a1300c81983` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:08.494016 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e39097-280e-425f-869a-fc0c01ae1c6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=362a6e9c40944d8e8d818a1300c81983)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:28:08.495954 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e39097-280e-425f-869a-fc0c01ae1c6e admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:28:08.540106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e5e39097-280e-425f-869a-fc0c01ae1c6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '362a6e9c40944d8e8d818a1300c81983'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:28:08.547112 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e5e39097-280e-425f-869a-fc0c01ae1c6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '362a6e9c40944d8e8d818a1300c81983'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:28:08.547804 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e5e39097-280e-425f-869a-fc0c01ae1c6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '362a6e9c40944d8e8d818a1300c81983'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:28:08.555474 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e5e39097-280e-425f-869a-fc0c01ae1c6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '362a6e9c40944d8e8d818a1300c81983'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:28:08.556070 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e5e39097-280e-425f-869a-fc0c01ae1c6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '362a6e9c40944d8e8d818a1300c81983'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:28:08.573376 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1413/2805] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:28:08 2022] DELETE /identity/v3/users/362a6e9c40944d8e8d818a1300c81983 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:28:08.761227 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddd4c6-8d6f-432d-80b9-c27d410fa458 service swift] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:28:08.772140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddd4c6-8d6f-432d-80b9-c27d410fa458 service swift] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:28:08.831094 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fe5c1-869d-4802-b70e-3cc4ae3da647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:28:08.832009 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fe5c1-869d-4802-b70e-3cc4ae3da647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:08.832289 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fe5c1-869d-4802-b70e-3cc4ae3da647 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:08.832597 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fe5c1-869d-4802-b70e-3cc4ae3da647 admin admin] PATH_INFO: `/v3/projects/438ea2009cc142578a97d17a081e8862` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:08.833895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894fe5c1-869d-4802-b70e-3cc4ae3da647 admin admin] RBAC: Authorizing `identity:delete_project(project_id=438ea2009cc142578a97d17a081e8862)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:28:08.835780 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894fe5c1-869d-4802-b70e-3cc4ae3da647 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:28:08.882242 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-894fe5c1-869d-4802-b70e-3cc4ae3da647 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '438ea2009cc142578a97d17a081e8862', 'request_id': 'req-894fe5c1-869d-4802-b70e-3cc4ae3da647'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:28:08.888206 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-894fe5c1-869d-4802-b70e-3cc4ae3da647 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '438ea2009cc142578a97d17a081e8862', 'request_id': 'req-894fe5c1-869d-4802-b70e-3cc4ae3da647'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:28:08.893538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-894fe5c1-869d-4802-b70e-3cc4ae3da647 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '438ea2009cc142578a97d17a081e8862', 'request_id': 'req-894fe5c1-869d-4802-b70e-3cc4ae3da647'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:28:08.906565 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-894fe5c1-869d-4802-b70e-3cc4ae3da647 admin admin] The token cache is being invalidate because project 438ea2009cc142578a97d17a081e8862 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:28:08.906853 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-894fe5c1-869d-4802-b70e-3cc4ae3da647 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:28:08.907455 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-894fe5c1-869d-4802-b70e-3cc4ae3da647 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:28:08.909025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1393/2806] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:28:08 2022] DELETE /identity/v3/projects/438ea2009cc142578a97d17a081e8862 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:28:08.913902 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03bf05a-616e-4eef-895a-03039e8bf8fe None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:08.914188 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03bf05a-616e-4eef-895a-03039e8bf8fe None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:08.914460 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03bf05a-616e-4eef-895a-03039e8bf8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:08.933217 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-c03bf05a-616e-4eef-895a-03039e8bf8fe None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:28:08.934419 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-c03bf05a-616e-4eef-895a-03039e8bf8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:28:08.996369 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1414/2807] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:28:09.004175 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1398d62-794f-470a-b9b1-0e5ffef55c32 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:09.004436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1398d62-794f-470a-b9b1-0e5ffef55c32 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:09.004739 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1398d62-794f-470a-b9b1-0e5ffef55c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:09.024459 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-a1398d62-794f-470a-b9b1-0e5ffef55c32 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:28:09.025664 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a1398d62-794f-470a-b9b1-0e5ffef55c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:28:09.035603 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1394/2808] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:28:09.040075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03bf05a-616e-4eef-895a-03039e8bf8fe None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:28:09.049793 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03bf05a-616e-4eef-895a-03039e8bf8fe None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:28:09.054631 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede538d0-e101-4e5d-a8cb-ae0ac0fe6a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:28:09.055280 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede538d0-e101-4e5d-a8cb-ae0ac0fe6a4a admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:09.055556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede538d0-e101-4e5d-a8cb-ae0ac0fe6a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:09.055824 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede538d0-e101-4e5d-a8cb-ae0ac0fe6a4a admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:09.056544 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede538d0-e101-4e5d-a8cb-ae0ac0fe6a4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 10:28:09.056881 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede538d0-e101-4e5d-a8cb-ae0ac0fe6a4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:28:09.058512 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede538d0-e101-4e5d-a8cb-ae0ac0fe6a4a admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:28:09.070575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1415/2809] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 10:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 10:28:09.075658 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1398d62-794f-470a-b9b1-0e5ffef55c32 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:28:09.086080 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1398d62-794f-470a-b9b1-0e5ffef55c32 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:28:09.092147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c4361-0f39-4a0c-bc7e-c10716232fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:28:09.092842 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c4361-0f39-4a0c-bc7e-c10716232fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:09.093120 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c4361-0f39-4a0c-bc7e-c10716232fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:09.093389 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c4361-0f39-4a0c-bc7e-c10716232fb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:09.094072 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c4361-0f39-4a0c-bc7e-c10716232fb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:28:09.095990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c4361-0f39-4a0c-bc7e-c10716232fb1 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:28:09.130083 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1395/2810] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:28:09 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:28:09.134862 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede538d0-e101-4e5d-a8cb-ae0ac0fe6a4a admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:28:09.145510 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede538d0-e101-4e5d-a8cb-ae0ac0fe6a4a admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:28:09.203469 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f608fa-f0a1-43fd-9461-1b0861dc5a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:28:09.204138 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f608fa-f0a1-43fd-9461-1b0861dc5a9d admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:09.204756 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f608fa-f0a1-43fd-9461-1b0861dc5a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:09.205030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f608fa-f0a1-43fd-9461-1b0861dc5a9d admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:09.205703 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f608fa-f0a1-43fd-9461-1b0861dc5a9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:28:09.207632 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f608fa-f0a1-43fd-9461-1b0861dc5a9d admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:28:09.228044 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1416/2811] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:28:09 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:28:09.232289 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c4361-0f39-4a0c-bc7e-c10716232fb1 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:28:09.242892 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c4361-0f39-4a0c-bc7e-c10716232fb1 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:28:09.248916 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74050422-9ea8-4de1-a7c2-d1cf4b4bbc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:28:09.249691 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74050422-9ea8-4de1-a7c2-d1cf4b4bbc7c admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:09.249969 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74050422-9ea8-4de1-a7c2-d1cf4b4bbc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:09.250235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74050422-9ea8-4de1-a7c2-d1cf4b4bbc7c admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:09.250964 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74050422-9ea8-4de1-a7c2-d1cf4b4bbc7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:28:09.252560 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74050422-9ea8-4de1-a7c2-d1cf4b4bbc7c admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:28:09.263627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1396/2812] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:28:09.267639 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f608fa-f0a1-43fd-9461-1b0861dc5a9d admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:28:09.278125 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f608fa-f0a1-43fd-9461-1b0861dc5a9d admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:28:09.283791 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcedfe-28e8-4011-af1e-3d15e196af8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:28:09.284445 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcedfe-28e8-4011-af1e-3d15e196af8c admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:09.284794 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcedfe-28e8-4011-af1e-3d15e196af8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:09.285059 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcedfe-28e8-4011-af1e-3d15e196af8c admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:09.285774 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcedfe-28e8-4011-af1e-3d15e196af8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:28:09.287383 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcedfe-28e8-4011-af1e-3d15e196af8c admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:28:09.298028 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1417/2813] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:28:09.302416 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74050422-9ea8-4de1-a7c2-d1cf4b4bbc7c admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:28:09.312545 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74050422-9ea8-4de1-a7c2-d1cf4b4bbc7c admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:28:09.317691 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e3f42-4ea2-4e06-865f-c66adace9db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:28:09.318518 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e3f42-4ea2-4e06-865f-c66adace9db2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:09.318791 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e3f42-4ea2-4e06-865f-c66adace9db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:09.319059 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e3f42-4ea2-4e06-865f-c66adace9db2 admin admin] PATH_INFO: `/v3/projects/b6f6a7d2d3144931bab8f38bbece1b8f/users/7ce0afdf478e412b882e33a96511a1a3/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:09.334436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e3f42-4ea2-4e06-865f-c66adace9db2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6f6a7d2d3144931bab8f38bbece1b8f, user_id=7ce0afdf478e412b882e33a96511a1a3, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:28:09.337111 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e3f42-4ea2-4e06-865f-c66adace9db2 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:28:09.352388 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1397/2814] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:28:09 2022] PUT /identity/v3/projects/b6f6a7d2d3144931bab8f38bbece1b8f/users/7ce0afdf478e412b882e33a96511a1a3/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:28:09.362462 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189614b3-2ad6-495a-9ed5-183919a7c54d None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:09.362749 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189614b3-2ad6-495a-9ed5-183919a7c54d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:09.363024 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189614b3-2ad6-495a-9ed5-183919a7c54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:09.393144 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-189614b3-2ad6-495a-9ed5-183919a7c54d None None] MFA Rules not processed for user `7ce0afdf478e412b882e33a96511a1a3`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:28:09.395066 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-189614b3-2ad6-495a-9ed5-183919a7c54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:28:09.479001 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1418/2815] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:28:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:28:09.492287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e3f42-4ea2-4e06-865f-c66adace9db2 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:28:09.494403 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-c79e3f42-4ea2-4e06-865f-c66adace9db2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:28:09.555888 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e3f42-4ea2-4e06-865f-c66adace9db2 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:28:09.560477 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b511e8-3ea3-409f-b859-27fd13622c36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:28:09.561157 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b511e8-3ea3-409f-b859-27fd13622c36 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:09.561436 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b511e8-3ea3-409f-b859-27fd13622c36 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:09.562067 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b511e8-3ea3-409f-b859-27fd13622c36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:09.570599 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b511e8-3ea3-409f-b859-27fd13622c36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:28:09.572047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b511e8-3ea3-409f-b859-27fd13622c36 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:28:09.581753 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1398/2816] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:28:09 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 10:28:49.881104 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189614b3-2ad6-495a-9ed5-183919a7c54d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:28:49.945223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189614b3-2ad6-495a-9ed5-183919a7c54d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:28:49.950087 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b314a-782c-4eb6-954a-60aac858b0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:28:49.950993 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b314a-782c-4eb6-954a-60aac858b0df admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:49.951269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b314a-782c-4eb6-954a-60aac858b0df admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:49.951536 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b314a-782c-4eb6-954a-60aac858b0df admin admin] PATH_INFO: `/v3/users/7ce0afdf478e412b882e33a96511a1a3` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:49.952775 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b314a-782c-4eb6-954a-60aac858b0df admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ce0afdf478e412b882e33a96511a1a3)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:28:49.954999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b314a-782c-4eb6-954a-60aac858b0df admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:28:50.006362 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e38b314a-782c-4eb6-954a-60aac858b0df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ce0afdf478e412b882e33a96511a1a3'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:28:50.013753 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e38b314a-782c-4eb6-954a-60aac858b0df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ce0afdf478e412b882e33a96511a1a3'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:28:50.014417 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e38b314a-782c-4eb6-954a-60aac858b0df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ce0afdf478e412b882e33a96511a1a3'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:28:50.023267 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e38b314a-782c-4eb6-954a-60aac858b0df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ce0afdf478e412b882e33a96511a1a3'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:28:50.023891 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e38b314a-782c-4eb6-954a-60aac858b0df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ce0afdf478e412b882e33a96511a1a3'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:28:50.048142 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1419/2817] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:28:49 2022] DELETE /identity/v3/users/7ce0afdf478e412b882e33a96511a1a3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:28:50.053109 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ccc60d-b990-4028-bc26-f79b7866662e None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:50.053627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ccc60d-b990-4028-bc26-f79b7866662e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:50.053722 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ccc60d-b990-4028-bc26-f79b7866662e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:50.074668 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-f4ccc60d-b990-4028-bc26-f79b7866662e None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:28:50.076223 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f4ccc60d-b990-4028-bc26-f79b7866662e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:28:50.148776 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1399/2818] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:28:50.158884 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b314a-782c-4eb6-954a-60aac858b0df admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:28:50.160582 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e38b314a-782c-4eb6-954a-60aac858b0df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:28:50.222515 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b314a-782c-4eb6-954a-60aac858b0df admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:28:50.227145 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68090005-b85b-4560-86e1-cf2e5c09573d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:28:50.227781 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68090005-b85b-4560-86e1-cf2e5c09573d service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:50.228059 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68090005-b85b-4560-86e1-cf2e5c09573d service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:50.228331 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68090005-b85b-4560-86e1-cf2e5c09573d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:50.238657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68090005-b85b-4560-86e1-cf2e5c09573d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:28:50.240118 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68090005-b85b-4560-86e1-cf2e5c09573d service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:28:50.249998 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1420/2819] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:28:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:28:50.529147 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ccc60d-b990-4028-bc26-f79b7866662e None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:28:50.530632 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f4ccc60d-b990-4028-bc26-f79b7866662e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:28:50.543261 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ccc60d-b990-4028-bc26-f79b7866662e None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:28:50.548765 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f18789-2015-4f65-8319-03d358aa3cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:28:50.549694 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f18789-2015-4f65-8319-03d358aa3cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:50.550038 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f18789-2015-4f65-8319-03d358aa3cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:50.550361 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f18789-2015-4f65-8319-03d358aa3cc6 admin admin] PATH_INFO: `/v3/projects/b6f6a7d2d3144931bab8f38bbece1b8f` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:50.551825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f18789-2015-4f65-8319-03d358aa3cc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6f6a7d2d3144931bab8f38bbece1b8f)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:28:50.553796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f18789-2015-4f65-8319-03d358aa3cc6 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:28:50.608314 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e3f18789-2015-4f65-8319-03d358aa3cc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6f6a7d2d3144931bab8f38bbece1b8f', 'request_id': 'req-e3f18789-2015-4f65-8319-03d358aa3cc6'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:28:50.614558 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e3f18789-2015-4f65-8319-03d358aa3cc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6f6a7d2d3144931bab8f38bbece1b8f', 'request_id': 'req-e3f18789-2015-4f65-8319-03d358aa3cc6'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:28:50.620688 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e3f18789-2015-4f65-8319-03d358aa3cc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6f6a7d2d3144931bab8f38bbece1b8f', 'request_id': 'req-e3f18789-2015-4f65-8319-03d358aa3cc6'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:28:50.635624 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e3f18789-2015-4f65-8319-03d358aa3cc6 admin admin] The token cache is being invalidate because project b6f6a7d2d3144931bab8f38bbece1b8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:28:50.635913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e3f18789-2015-4f65-8319-03d358aa3cc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:28:50.636579 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e3f18789-2015-4f65-8319-03d358aa3cc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:28:50.638253 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1400/2820] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:28:50 2022] DELETE /identity/v3/projects/b6f6a7d2d3144931bab8f38bbece1b8f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:28:50.644448 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d248949-2a51-4f20-9a59-0d11e7dff40d None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:50.645020 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d248949-2a51-4f20-9a59-0d11e7dff40d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:50.645499 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d248949-2a51-4f20-9a59-0d11e7dff40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:50.671861 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-7d248949-2a51-4f20-9a59-0d11e7dff40d None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:28:50.673471 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-7d248949-2a51-4f20-9a59-0d11e7dff40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:28:50.745803 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1421/2821] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 10:28:50.753775 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba92427-37e3-4193-8dcd-6161d55236d4 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:50.754053 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba92427-37e3-4193-8dcd-6161d55236d4 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:50.754318 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba92427-37e3-4193-8dcd-6161d55236d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:50.773127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-bba92427-37e3-4193-8dcd-6161d55236d4 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:28:50.775858 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-bba92427-37e3-4193-8dcd-6161d55236d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:28:50.783065 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1401/2822] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:28:50.789820 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d248949-2a51-4f20-9a59-0d11e7dff40d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:28:50.805430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d248949-2a51-4f20-9a59-0d11e7dff40d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:28:50.811766 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f9ba1-bb64-4ecb-931f-e52b75ed3a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:28:50.812431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f9ba1-bb64-4ecb-931f-e52b75ed3a64 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:50.812721 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f9ba1-bb64-4ecb-931f-e52b75ed3a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:50.813012 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f9ba1-bb64-4ecb-931f-e52b75ed3a64 admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:50.813745 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f9ba1-bb64-4ecb-931f-e52b75ed3a64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 10:28:50.814050 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f9ba1-bb64-4ecb-931f-e52b75ed3a64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:28:50.815665 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f9ba1-bb64-4ecb-931f-e52b75ed3a64 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:28:50.827689 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1422/2823] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 10:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 10:28:50.832588 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba92427-37e3-4193-8dcd-6161d55236d4 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:28:50.842040 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba92427-37e3-4193-8dcd-6161d55236d4 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:28:50.847157 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e76e0-7c58-46e8-bd97-a5e22bc1e9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:28:50.847803 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e76e0-7c58-46e8-bd97-a5e22bc1e9ee admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:50.848079 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e76e0-7c58-46e8-bd97-a5e22bc1e9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:50.848346 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e76e0-7c58-46e8-bd97-a5e22bc1e9ee admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:50.849077 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e76e0-7c58-46e8-bd97-a5e22bc1e9ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:28:50.851007 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e76e0-7c58-46e8-bd97-a5e22bc1e9ee admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:28:50.878819 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1402/2824] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:28:50 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:28:50.883096 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f9ba1-bb64-4ecb-931f-e52b75ed3a64 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:28:50.893394 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f9ba1-bb64-4ecb-931f-e52b75ed3a64 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:28:50.949281 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73720b-a3d4-4ec2-abc0-12b109ebb1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:28:50.949872 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73720b-a3d4-4ec2-abc0-12b109ebb1cd admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:50.950148 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73720b-a3d4-4ec2-abc0-12b109ebb1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:50.950407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73720b-a3d4-4ec2-abc0-12b109ebb1cd admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:50.951106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a73720b-a3d4-4ec2-abc0-12b109ebb1cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:28:50.953051 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a73720b-a3d4-4ec2-abc0-12b109ebb1cd admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:28:50.974587 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1423/2825] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:28:50 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:28:50.979002 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e76e0-7c58-46e8-bd97-a5e22bc1e9ee admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:28:50.990105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e76e0-7c58-46e8-bd97-a5e22bc1e9ee admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:28:50.995015 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7201557-5576-40ab-898e-2f8ff2de7e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:28:50.995679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7201557-5576-40ab-898e-2f8ff2de7e0a admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:50.995952 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7201557-5576-40ab-898e-2f8ff2de7e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:50.996216 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7201557-5576-40ab-898e-2f8ff2de7e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:50.996973 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7201557-5576-40ab-898e-2f8ff2de7e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:28:50.998575 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7201557-5576-40ab-898e-2f8ff2de7e0a admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:28:51.009903 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1403/2826] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:28:51.013806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73720b-a3d4-4ec2-abc0-12b109ebb1cd admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:28:51.024156 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73720b-a3d4-4ec2-abc0-12b109ebb1cd admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:28:51.028956 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78848d-0aef-440b-b2fb-33b143fdb333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:28:51.029620 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78848d-0aef-440b-b2fb-33b143fdb333 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:51.029901 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78848d-0aef-440b-b2fb-33b143fdb333 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:51.030166 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78848d-0aef-440b-b2fb-33b143fdb333 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:51.030904 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e78848d-0aef-440b-b2fb-33b143fdb333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:28:51.032482 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e78848d-0aef-440b-b2fb-33b143fdb333 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:28:51.042658 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1424/2827] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:28:51.050205 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7201557-5576-40ab-898e-2f8ff2de7e0a admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:28:51.062623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7201557-5576-40ab-898e-2f8ff2de7e0a admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:28:51.069214 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6734b5-35ca-44f4-94e9-257de44ece36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:28:51.070415 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6734b5-35ca-44f4-94e9-257de44ece36 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:51.070778 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6734b5-35ca-44f4-94e9-257de44ece36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:51.071141 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6734b5-35ca-44f4-94e9-257de44ece36 admin admin] PATH_INFO: `/v3/projects/a3e0d3bb02404e219e41179b1f7784da/users/27ff738b86d3437082d1c00f97f0f0e5/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:51.086707 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6734b5-35ca-44f4-94e9-257de44ece36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3e0d3bb02404e219e41179b1f7784da, user_id=27ff738b86d3437082d1c00f97f0f0e5, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:28:51.089421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6734b5-35ca-44f4-94e9-257de44ece36 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:28:51.103451 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1404/2828] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:28:51 2022] PUT /identity/v3/projects/a3e0d3bb02404e219e41179b1f7784da/users/27ff738b86d3437082d1c00f97f0f0e5/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:28:51.114113 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a918e-d088-46bf-b72b-6a77768d873b None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:51.114399 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a918e-d088-46bf-b72b-6a77768d873b None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:51.114670 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a918e-d088-46bf-b72b-6a77768d873b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:51.145450 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-988a918e-d088-46bf-b72b-6a77768d873b None None] MFA Rules not processed for user `27ff738b86d3437082d1c00f97f0f0e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:28:51.146631 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-988a918e-d088-46bf-b72b-6a77768d873b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:28:51.221193 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1425/2829] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:28:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:28:51.232995 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6734b5-35ca-44f4-94e9-257de44ece36 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:28:51.234941 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-1e6734b5-35ca-44f4-94e9-257de44ece36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:28:51.294171 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6734b5-35ca-44f4-94e9-257de44ece36 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:28:51.298374 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb813d51-40cb-4806-b785-7f955e032dc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:28:51.299022 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb813d51-40cb-4806-b785-7f955e032dc8 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:51.299295 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb813d51-40cb-4806-b785-7f955e032dc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:51.299570 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb813d51-40cb-4806-b785-7f955e032dc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:51.308638 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb813d51-40cb-4806-b785-7f955e032dc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:28:51.310101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb813d51-40cb-4806-b785-7f955e032dc8 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:28:51.318953 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1405/2830] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:28:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:28:58.017504 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a918e-d088-46bf-b72b-6a77768d873b None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:28:58.019690 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-988a918e-d088-46bf-b72b-6a77768d873b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:28:58.090007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a918e-d088-46bf-b72b-6a77768d873b None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:28:58.093541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11fd4f-0feb-4469-86f5-253daeafbe66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:28:58.094159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11fd4f-0feb-4469-86f5-253daeafbe66 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:28:58.094461 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11fd4f-0feb-4469-86f5-253daeafbe66 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:28:58.094733 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11fd4f-0feb-4469-86f5-253daeafbe66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:28:58.096687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-4b11fd4f-0feb-4469-86f5-253daeafbe66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:28:58.107108 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b11fd4f-0feb-4469-86f5-253daeafbe66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:28:58.109010 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b11fd4f-0feb-4469-86f5-253daeafbe66 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:28:58.119901 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1426/2831] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:28:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:29:01.151937 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb813d51-40cb-4806-b785-7f955e032dc8 service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:29:01.218151 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb813d51-40cb-4806-b785-7f955e032dc8 service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:29:01.223350 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544121e-7126-49f9-bf5c-e7904ca69972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:29:01.224228 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544121e-7126-49f9-bf5c-e7904ca69972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:29:01.224497 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544121e-7126-49f9-bf5c-e7904ca69972 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:29:01.224796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544121e-7126-49f9-bf5c-e7904ca69972 admin admin] PATH_INFO: `/v3/users/27ff738b86d3437082d1c00f97f0f0e5` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:29:01.226035 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6544121e-7126-49f9-bf5c-e7904ca69972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27ff738b86d3437082d1c00f97f0f0e5)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:29:01.227957 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6544121e-7126-49f9-bf5c-e7904ca69972 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:29:01.279278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6544121e-7126-49f9-bf5c-e7904ca69972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27ff738b86d3437082d1c00f97f0f0e5'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:29:01.286519 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6544121e-7126-49f9-bf5c-e7904ca69972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27ff738b86d3437082d1c00f97f0f0e5'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:29:01.294677 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6544121e-7126-49f9-bf5c-e7904ca69972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27ff738b86d3437082d1c00f97f0f0e5'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:29:01.295284 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6544121e-7126-49f9-bf5c-e7904ca69972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27ff738b86d3437082d1c00f97f0f0e5'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:29:01.295955 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-6544121e-7126-49f9-bf5c-e7904ca69972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27ff738b86d3437082d1c00f97f0f0e5'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:29:01.313073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1406/2832] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:29:01 2022] DELETE /identity/v3/users/27ff738b86d3437082d1c00f97f0f0e5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:29:01.317835 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603e44e-2fa6-4d65-88f2-c0f7e35722ea None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:29:01.318349 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603e44e-2fa6-4d65-88f2-c0f7e35722ea None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:29:01.318446 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603e44e-2fa6-4d65-88f2-c0f7e35722ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:29:01.338349 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-e603e44e-2fa6-4d65-88f2-c0f7e35722ea None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:29:01.340029 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e603e44e-2fa6-4d65-88f2-c0f7e35722ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:29:01.406753 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1427/2833] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 10:29:01.416822 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544121e-7126-49f9-bf5c-e7904ca69972 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:29:01.418246 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-6544121e-7126-49f9-bf5c-e7904ca69972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:29:01.493127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544121e-7126-49f9-bf5c-e7904ca69972 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:29:01.497712 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3ca3f-3b4c-4cbe-9649-29677a5cb049 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:29:01.498367 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3ca3f-3b4c-4cbe-9649-29677a5cb049 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:29:01.498623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3ca3f-3b4c-4cbe-9649-29677a5cb049 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:29:01.498889 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3ca3f-3b4c-4cbe-9649-29677a5cb049 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:29:01.511837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a3ca3f-3b4c-4cbe-9649-29677a5cb049 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:29:01.513313 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a3ca3f-3b4c-4cbe-9649-29677a5cb049 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:29:01.523484 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1407/2834] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:29:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:29:01.830597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e603e44e-2fa6-4d65-88f2-c0f7e35722ea None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:29:01.832021 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e603e44e-2fa6-4d65-88f2-c0f7e35722ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:29:01.845414 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e603e44e-2fa6-4d65-88f2-c0f7e35722ea None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:29:01.850641 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d0057-4884-4320-8160-c6d5f3b48507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:29:01.852157 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d0057-4884-4320-8160-c6d5f3b48507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:29:01.852466 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d0057-4884-4320-8160-c6d5f3b48507 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:29:01.853052 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d0057-4884-4320-8160-c6d5f3b48507 admin admin] PATH_INFO: `/v3/projects/a3e0d3bb02404e219e41179b1f7784da` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:29:01.854372 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9d0057-4884-4320-8160-c6d5f3b48507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3e0d3bb02404e219e41179b1f7784da)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:29:01.856390 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9d0057-4884-4320-8160-c6d5f3b48507 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:29:01.904885 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-1c9d0057-4884-4320-8160-c6d5f3b48507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3e0d3bb02404e219e41179b1f7784da', 'request_id': 'req-1c9d0057-4884-4320-8160-c6d5f3b48507'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:29:01.910892 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-1c9d0057-4884-4320-8160-c6d5f3b48507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3e0d3bb02404e219e41179b1f7784da', 'request_id': 'req-1c9d0057-4884-4320-8160-c6d5f3b48507'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:29:01.916395 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-1c9d0057-4884-4320-8160-c6d5f3b48507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3e0d3bb02404e219e41179b1f7784da', 'request_id': 'req-1c9d0057-4884-4320-8160-c6d5f3b48507'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:29:01.930525 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-1c9d0057-4884-4320-8160-c6d5f3b48507 admin admin] The token cache is being invalidate because project a3e0d3bb02404e219e41179b1f7784da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:29:01.930806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-1c9d0057-4884-4320-8160-c6d5f3b48507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:29:01.931482 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-1c9d0057-4884-4320-8160-c6d5f3b48507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:29:01.933173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1428/2835] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:29:01 2022] DELETE /identity/v3/projects/a3e0d3bb02404e219e41179b1f7784da => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:29:08.852899 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3ca3f-3b4c-4cbe-9649-29677a5cb049 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:29:08.854525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-91a3ca3f-3b4c-4cbe-9649-29677a5cb049 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:29:08.914135 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3ca3f-3b4c-4cbe-9649-29677a5cb049 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:29:08.964970 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15930ba3-f49f-4256-b21d-2bdea0b2af52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:29:08.965625 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15930ba3-f49f-4256-b21d-2bdea0b2af52 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:29:08.965894 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15930ba3-f49f-4256-b21d-2bdea0b2af52 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:29:08.966159 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15930ba3-f49f-4256-b21d-2bdea0b2af52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:29:08.968161 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-15930ba3-f49f-4256-b21d-2bdea0b2af52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:29:09.038418 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15930ba3-f49f-4256-b21d-2bdea0b2af52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:29:09.039896 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15930ba3-f49f-4256-b21d-2bdea0b2af52 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:29:09.059154 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1408/2836] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:29:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:30:00.120521 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b363a05-ed32-4838-b5c6-fe6de3753d9d None None] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:00.121410 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b363a05-ed32-4838-b5c6-fe6de3753d9d None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:00.121735 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b363a05-ed32-4838-b5c6-fe6de3753d9d None None] PATH_INFO: `/` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:00.124741 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1429/2837] 10.222.0.26 () {54 vars in 921 bytes} [Mon May 9 10:30:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 10:30:00.135447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ad77d-370a-46f0-8717-5770eb257a79 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:00.136344 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ad77d-370a-46f0-8717-5770eb257a79 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:00.136492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ad77d-370a-46f0-8717-5770eb257a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:00.205922 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-d51ad77d-370a-46f0-8717-5770eb257a79 None None] MFA Rules not processed for user `0f0d3f4264a04093894d6bc5c5c45c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:30:00.207988 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-d51ad77d-370a-46f0-8717-5770eb257a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:30:00.300562 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1409/2838] 10.222.0.26 () {58 vars in 1018 bytes} [Mon May 9 10:30:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:30:00.317516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b363a05-ed32-4838-b5c6-fe6de3753d9d None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:30:00.322109 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-8b363a05-ed32-4838-b5c6-fe6de3753d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:30:00.341012 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b363a05-ed32-4838-b5c6-fe6de3753d9d None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:30:00.345201 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03271fe9-b6e3-4b0c-8e6a-c998822904b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:30:00.345938 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03271fe9-b6e3-4b0c-8e6a-c998822904b6 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:00.346342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03271fe9-b6e3-4b0c-8e6a-c998822904b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:00.346692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03271fe9-b6e3-4b0c-8e6a-c998822904b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:00.357430 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03271fe9-b6e3-4b0c-8e6a-c998822904b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:30:00.359085 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03271fe9-b6e3-4b0c-8e6a-c998822904b6 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:30:00.374592 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1430/2839] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:30:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:30:00.480039 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2109f7e-0212-4a44-a50b-dd96c9a2a54e None None] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:00.480526 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2109f7e-0212-4a44-a50b-dd96c9a2a54e None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:00.481067 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2109f7e-0212-4a44-a50b-dd96c9a2a54e None None] PATH_INFO: `/` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:00.483900 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1410/2840] 10.222.0.26 () {54 vars in 921 bytes} [Mon May 9 10:30:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 10:30:00.492047 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9fcfa-2deb-45f5-9cdf-42f07caa2162 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:00.492617 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9fcfa-2deb-45f5-9cdf-42f07caa2162 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:00.493204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9fcfa-2deb-45f5-9cdf-42f07caa2162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:00.555081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-cbc9fcfa-2deb-45f5-9cdf-42f07caa2162 None None] MFA Rules not processed for user `6362303cb918486c8525225914332ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:30:00.558466 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-cbc9fcfa-2deb-45f5-9cdf-42f07caa2162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:30:00.663018 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1431/2841] 10.222.0.26 () {58 vars in 1018 bytes} [Mon May 9 10:30:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:30:00.675287 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2109f7e-0212-4a44-a50b-dd96c9a2a54e None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:30:00.687802 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2109f7e-0212-4a44-a50b-dd96c9a2a54e None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:30:00.692013 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb15b8-636a-4408-8052-92787b114322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:30:00.692806 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb15b8-636a-4408-8052-92787b114322 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:00.693221 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb15b8-636a-4408-8052-92787b114322 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:00.693617 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb15b8-636a-4408-8052-92787b114322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:00.706538 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb15b8-636a-4408-8052-92787b114322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:30:00.708357 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb15b8-636a-4408-8052-92787b114322 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:30:00.721710 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1411/2842] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:30:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 10:30:00.790970 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9fcfa-2deb-45f5-9cdf-42f07caa2162 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:30:00.792397 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-cbc9fcfa-2deb-45f5-9cdf-42f07caa2162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:30:00.862429 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9fcfa-2deb-45f5-9cdf-42f07caa2162 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:30:00.867732 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f39831-231a-4616-a50b-cdee672bfaca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9e458c2c5f4ed1b3eddd84c2ec1703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:30:00.868393 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f39831-231a-4616-a50b-cdee672bfaca service placement] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:00.868694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f39831-231a-4616-a50b-cdee672bfaca service placement] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:00.868980 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f39831-231a-4616-a50b-cdee672bfaca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:00.870875 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-95f39831-231a-4616-a50b-cdee672bfaca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:30:00.884252 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f39831-231a-4616-a50b-cdee672bfaca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:30:00.886184 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f39831-231a-4616-a50b-cdee672bfaca service placement] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:30:00.897371 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1432/2843] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:30:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:30:02.063577 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb15b8-636a-4408-8052-92787b114322 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:30:02.065990 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a6fb15b8-636a-4408-8052-92787b114322 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:30:02.090958 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb15b8-636a-4408-8052-92787b114322 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:30:02.097925 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc364e21-01be-4eff-88ec-68e1ac6b0fa5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:30:02.098789 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc364e21-01be-4eff-88ec-68e1ac6b0fa5 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:02.099167 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc364e21-01be-4eff-88ec-68e1ac6b0fa5 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:02.099551 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc364e21-01be-4eff-88ec-68e1ac6b0fa5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:02.102618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-bc364e21-01be-4eff-88ec-68e1ac6b0fa5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:30:02.122306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc364e21-01be-4eff-88ec-68e1ac6b0fa5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:30:02.124351 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc364e21-01be-4eff-88ec-68e1ac6b0fa5 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:30:02.145431 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1412/2844] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:30:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:30:07.850741 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f39831-231a-4616-a50b-cdee672bfaca service placement] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:30:07.879761 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f39831-231a-4616-a50b-cdee672bfaca service placement] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:30:07.889535 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b9113-678c-44ab-9a9a-7390a2c285af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:30:07.890335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b9113-678c-44ab-9a9a-7390a2c285af service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:07.890780 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b9113-678c-44ab-9a9a-7390a2c285af service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:07.891152 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b9113-678c-44ab-9a9a-7390a2c285af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:07.893561 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-816b9113-678c-44ab-9a9a-7390a2c285af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:30:07.909331 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b9113-678c-44ab-9a9a-7390a2c285af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:30:07.911815 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b9113-678c-44ab-9a9a-7390a2c285af service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:30:07.930309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1433/2845] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:30:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:30:08.747851 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc364e21-01be-4eff-88ec-68e1ac6b0fa5 service nova] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:30:08.750405 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-bc364e21-01be-4eff-88ec-68e1ac6b0fa5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:30:08.788651 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc364e21-01be-4eff-88ec-68e1ac6b0fa5 service nova] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:30:08.795117 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa9a35-ff26-4c43-ade0-5a5d7debdd95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:30:08.796128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa9a35-ff26-4c43-ade0-5a5d7debdd95 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:08.796594 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa9a35-ff26-4c43-ade0-5a5d7debdd95 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:08.797082 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa9a35-ff26-4c43-ade0-5a5d7debdd95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:08.799978 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-92fa9a35-ff26-4c43-ade0-5a5d7debdd95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:30:08.859646 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa9a35-ff26-4c43-ade0-5a5d7debdd95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:30:08.862264 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa9a35-ff26-4c43-ade0-5a5d7debdd95 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:30:08.881128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1413/2846] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:30:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:30:11.520873 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b9113-678c-44ab-9a9a-7390a2c285af service nova] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:30:11.545043 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b9113-678c-44ab-9a9a-7390a2c285af service nova] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:30:11.549722 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf47d3-8711-4332-9e28-b56f746ea0a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:30:11.550593 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf47d3-8711-4332-9e28-b56f746ea0a1 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:11.550927 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf47d3-8711-4332-9e28-b56f746ea0a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:11.551331 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf47d3-8711-4332-9e28-b56f746ea0a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:11.553640 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-6daf47d3-8711-4332-9e28-b56f746ea0a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:30:11.583693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf47d3-8711-4332-9e28-b56f746ea0a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:30:11.585226 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf47d3-8711-4332-9e28-b56f746ea0a1 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:30:11.597679 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1434/2847] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:30:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 10:30:12.740517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa9a35-ff26-4c43-ade0-5a5d7debdd95 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:30:12.763313 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa9a35-ff26-4c43-ade0-5a5d7debdd95 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:30:12.769706 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97152a-100c-43c9-b57f-07fc2593e116 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:30:12.770823 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97152a-100c-43c9-b57f-07fc2593e116 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:12.771286 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97152a-100c-43c9-b57f-07fc2593e116 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:12.771755 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97152a-100c-43c9-b57f-07fc2593e116 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:12.775106 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-ab97152a-100c-43c9-b57f-07fc2593e116 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:30:12.865665 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab97152a-100c-43c9-b57f-07fc2593e116 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:30:12.867586 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab97152a-100c-43c9-b57f-07fc2593e116 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:30:12.882035 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1414/2848] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:30:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 10:30:17.304807 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf47d3-8711-4332-9e28-b56f746ea0a1 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:30:17.307067 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-6daf47d3-8711-4332-9e28-b56f746ea0a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:30:17.334655 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf47d3-8711-4332-9e28-b56f746ea0a1 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:30:17.343670 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04616775-9874-4a3f-9552-82ad83587227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:30:17.345418 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04616775-9874-4a3f-9552-82ad83587227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:17.346057 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04616775-9874-4a3f-9552-82ad83587227 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:17.346521 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04616775-9874-4a3f-9552-82ad83587227 admin admin] PATH_INFO: `/v3/users/e58c14dca7da4e6495f7e08f2bb127e2` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:17.348903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04616775-9874-4a3f-9552-82ad83587227 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e58c14dca7da4e6495f7e08f2bb127e2)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:30:17.352694 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04616775-9874-4a3f-9552-82ad83587227 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:30:17.422741 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-04616775-9874-4a3f-9552-82ad83587227 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e58c14dca7da4e6495f7e08f2bb127e2'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:30:17.430251 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-04616775-9874-4a3f-9552-82ad83587227 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e58c14dca7da4e6495f7e08f2bb127e2'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:30:17.430999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-04616775-9874-4a3f-9552-82ad83587227 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e58c14dca7da4e6495f7e08f2bb127e2'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:30:17.440171 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-04616775-9874-4a3f-9552-82ad83587227 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e58c14dca7da4e6495f7e08f2bb127e2'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:30:17.440873 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-04616775-9874-4a3f-9552-82ad83587227 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e58c14dca7da4e6495f7e08f2bb127e2'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:30:17.465173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1435/2849] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:30:17 2022] DELETE /identity/v3/users/e58c14dca7da4e6495f7e08f2bb127e2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:30:17.668239 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97152a-100c-43c9-b57f-07fc2593e116 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:30:17.683451 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97152a-100c-43c9-b57f-07fc2593e116 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:30:17.749434 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc549b83-ed6d-4dd9-9c7e-02e623645a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:30:17.750313 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc549b83-ed6d-4dd9-9c7e-02e623645a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:17.750621 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc549b83-ed6d-4dd9-9c7e-02e623645a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:17.750889 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc549b83-ed6d-4dd9-9c7e-02e623645a81 admin admin] PATH_INFO: `/v3/projects/0511fef4fd9f4bd09a3b1867e8556a06` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:17.752300 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc549b83-ed6d-4dd9-9c7e-02e623645a81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0511fef4fd9f4bd09a3b1867e8556a06)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:30:17.754370 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc549b83-ed6d-4dd9-9c7e-02e623645a81 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:30:17.805618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bc549b83-ed6d-4dd9-9c7e-02e623645a81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0511fef4fd9f4bd09a3b1867e8556a06', 'request_id': 'req-bc549b83-ed6d-4dd9-9c7e-02e623645a81'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:30:17.813713 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bc549b83-ed6d-4dd9-9c7e-02e623645a81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0511fef4fd9f4bd09a3b1867e8556a06', 'request_id': 'req-bc549b83-ed6d-4dd9-9c7e-02e623645a81'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:30:17.820086 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bc549b83-ed6d-4dd9-9c7e-02e623645a81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0511fef4fd9f4bd09a3b1867e8556a06', 'request_id': 'req-bc549b83-ed6d-4dd9-9c7e-02e623645a81'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:30:17.836072 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bc549b83-ed6d-4dd9-9c7e-02e623645a81 admin admin] The token cache is being invalidate because project 0511fef4fd9f4bd09a3b1867e8556a06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:30:17.836439 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bc549b83-ed6d-4dd9-9c7e-02e623645a81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:30:17.837403 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-bc549b83-ed6d-4dd9-9c7e-02e623645a81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:30:17.839255 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1415/2850] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:30:17 2022] DELETE /identity/v3/projects/0511fef4fd9f4bd09a3b1867e8556a06 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:30:17.847657 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa736eff-443a-4eb2-a728-18f84ba1a481 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:17.847988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa736eff-443a-4eb2-a728-18f84ba1a481 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:17.848309 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa736eff-443a-4eb2-a728-18f84ba1a481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:17.871515 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-aa736eff-443a-4eb2-a728-18f84ba1a481 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:30:17.872832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-aa736eff-443a-4eb2-a728-18f84ba1a481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:30:17.945216 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1436/2851] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:30:17.953973 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f992557-8126-42cb-8eac-208e011c4587 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:17.954246 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f992557-8126-42cb-8eac-208e011c4587 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:17.954547 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f992557-8126-42cb-8eac-208e011c4587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:17.974480 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-9f992557-8126-42cb-8eac-208e011c4587 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:30:17.975723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-9f992557-8126-42cb-8eac-208e011c4587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:30:17.984592 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1416/2852] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:30:17.989412 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa736eff-443a-4eb2-a728-18f84ba1a481 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:30:18.000449 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa736eff-443a-4eb2-a728-18f84ba1a481 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:30:18.005245 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc398b7-8011-4616-9167-c02b0b5c9573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:30:18.005945 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc398b7-8011-4616-9167-c02b0b5c9573 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:18.006222 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc398b7-8011-4616-9167-c02b0b5c9573 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:18.006495 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc398b7-8011-4616-9167-c02b0b5c9573 admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:18.007267 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc398b7-8011-4616-9167-c02b0b5c9573 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 10:30:18.007568 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc398b7-8011-4616-9167-c02b0b5c9573 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:30:18.009258 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc398b7-8011-4616-9167-c02b0b5c9573 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:30:18.022398 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1437/2853] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 10:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 10:30:18.027170 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f992557-8126-42cb-8eac-208e011c4587 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:30:18.037904 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f992557-8126-42cb-8eac-208e011c4587 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:30:18.043457 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ccc65-c90c-427f-8eae-0864963815f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:30:18.044210 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ccc65-c90c-427f-8eae-0864963815f3 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:18.044540 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ccc65-c90c-427f-8eae-0864963815f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:18.044988 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ccc65-c90c-427f-8eae-0864963815f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:18.045962 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ccc65-c90c-427f-8eae-0864963815f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:30:18.048562 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ccc65-c90c-427f-8eae-0864963815f3 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:30:18.080011 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1417/2854] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:30:18 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:30:18.084939 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc398b7-8011-4616-9167-c02b0b5c9573 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:30:18.097106 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc398b7-8011-4616-9167-c02b0b5c9573 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:30:18.164188 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367a422-79c3-4227-a98d-768fe332cd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:30:18.165072 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367a422-79c3-4227-a98d-768fe332cd48 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:18.165373 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367a422-79c3-4227-a98d-768fe332cd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:18.165782 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367a422-79c3-4227-a98d-768fe332cd48 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:18.166706 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5367a422-79c3-4227-a98d-768fe332cd48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:30:18.168775 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5367a422-79c3-4227-a98d-768fe332cd48 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:30:18.192550 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1438/2855] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:30:18 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:30:18.197129 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ccc65-c90c-427f-8eae-0864963815f3 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:30:18.208525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ccc65-c90c-427f-8eae-0864963815f3 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:30:18.214086 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7befcc8-a7bb-4f0a-bd8f-459df94109ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:30:18.214773 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7befcc8-a7bb-4f0a-bd8f-459df94109ee admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:18.215047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7befcc8-a7bb-4f0a-bd8f-459df94109ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:18.215314 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7befcc8-a7bb-4f0a-bd8f-459df94109ee admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:18.216047 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7befcc8-a7bb-4f0a-bd8f-459df94109ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:30:18.217713 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7befcc8-a7bb-4f0a-bd8f-459df94109ee admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:30:18.230003 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1418/2856] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:30:18.235116 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367a422-79c3-4227-a98d-768fe332cd48 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:30:18.246851 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367a422-79c3-4227-a98d-768fe332cd48 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:30:18.252007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6876e2-aae7-4546-9a7d-057e4a530e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:30:18.252735 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6876e2-aae7-4546-9a7d-057e4a530e74 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:18.253020 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6876e2-aae7-4546-9a7d-057e4a530e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:18.253291 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6876e2-aae7-4546-9a7d-057e4a530e74 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:18.254045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6876e2-aae7-4546-9a7d-057e4a530e74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:30:18.255687 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6876e2-aae7-4546-9a7d-057e4a530e74 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:30:18.266769 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1439/2857] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:30:18.271558 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7befcc8-a7bb-4f0a-bd8f-459df94109ee admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:30:18.283139 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7befcc8-a7bb-4f0a-bd8f-459df94109ee admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:30:18.289711 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf6556-c226-43b1-aa03-fa6bc49e5a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:30:18.290624 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf6556-c226-43b1-aa03-fa6bc49e5a5d admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:18.290911 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf6556-c226-43b1-aa03-fa6bc49e5a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:18.291172 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf6556-c226-43b1-aa03-fa6bc49e5a5d admin admin] PATH_INFO: `/v3/projects/6747c07e670e4379874273bc5a88397f/users/a5732efd025843a28c9b6948b4b893f5/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:18.312856 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf6556-c226-43b1-aa03-fa6bc49e5a5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6747c07e670e4379874273bc5a88397f, user_id=a5732efd025843a28c9b6948b4b893f5, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:30:18.315577 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf6556-c226-43b1-aa03-fa6bc49e5a5d admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:30:18.331553 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1419/2858] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:30:18 2022] PUT /identity/v3/projects/6747c07e670e4379874273bc5a88397f/users/a5732efd025843a28c9b6948b4b893f5/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:30:18.337504 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179858b9-4ea6-482b-8041-5f831d4cf5f8 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:18.338379 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179858b9-4ea6-482b-8041-5f831d4cf5f8 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:18.338379 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179858b9-4ea6-482b-8041-5f831d4cf5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:18.359307 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-179858b9-4ea6-482b-8041-5f831d4cf5f8 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:30:18.361075 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-179858b9-4ea6-482b-8041-5f831d4cf5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:30:18.428436 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1440/2859] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:30:18.440741 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf6556-c226-43b1-aa03-fa6bc49e5a5d admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:30:18.442677 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-cadf6556-c226-43b1-aa03-fa6bc49e5a5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:30:18.504698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf6556-c226-43b1-aa03-fa6bc49e5a5d admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:30:18.508691 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb86653-c217-4469-a9a9-c428b96a41ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:30:18.509396 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb86653-c217-4469-a9a9-c428b96a41ba service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:18.509683 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb86653-c217-4469-a9a9-c428b96a41ba service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:18.509954 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb86653-c217-4469-a9a9-c428b96a41ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:18.520968 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb86653-c217-4469-a9a9-c428b96a41ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:30:18.522577 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb86653-c217-4469-a9a9-c428b96a41ba service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:30:18.534376 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1420/2860] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:30:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:30:23.767332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ca839-df65-4bfb-90e7-9217c6f5e966 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:23.768248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ca839-df65-4bfb-90e7-9217c6f5e966 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:23.768248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ca839-df65-4bfb-90e7-9217c6f5e966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:23.808255 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-e08ca839-df65-4bfb-90e7-9217c6f5e966 None None] MFA Rules not processed for user `a5732efd025843a28c9b6948b4b893f5`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:30:23.809695 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e08ca839-df65-4bfb-90e7-9217c6f5e966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:30:23.908612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1441/2861] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:30:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:30:23.923296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb86653-c217-4469-a9a9-c428b96a41ba service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:30:23.925838 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-ffb86653-c217-4469-a9a9-c428b96a41ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:30:24.067134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb86653-c217-4469-a9a9-c428b96a41ba service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:30:24.073815 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682efd7-2ee2-41c5-b333-c494ec59b3be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:30:24.074590 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682efd7-2ee2-41c5-b333-c494ec59b3be service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:24.074987 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682efd7-2ee2-41c5-b333-c494ec59b3be service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:24.075493 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682efd7-2ee2-41c5-b333-c494ec59b3be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:24.086623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682efd7-2ee2-41c5-b333-c494ec59b3be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:30:24.089173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682efd7-2ee2-41c5-b333-c494ec59b3be service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:30:24.102048 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1421/2862] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:30:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 10:30:26.595818 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875938f-02b8-4400-97e0-6b865972cbca None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:26.597028 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875938f-02b8-4400-97e0-6b865972cbca None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:26.597426 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875938f-02b8-4400-97e0-6b865972cbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:26.619249 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-d875938f-02b8-4400-97e0-6b865972cbca None None] MFA Rules not processed for user `a5732efd025843a28c9b6948b4b893f5`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:30:26.620548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-d875938f-02b8-4400-97e0-6b865972cbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:30:26.628332 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1442/2863] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:30:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:30:56.360337 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682efd7-2ee2-41c5-b333-c494ec59b3be service nova] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:30:56.362029 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-a682efd7-2ee2-41c5-b333-c494ec59b3be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:30:56.462840 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682efd7-2ee2-41c5-b333-c494ec59b3be service nova] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:30:56.468213 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1336a-0408-487a-9d85-85ad2cdc8713 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9e458c2c5f4ed1b3eddd84c2ec1703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:30:56.468913 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1336a-0408-487a-9d85-85ad2cdc8713 service placement] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:30:56.469188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1336a-0408-487a-9d85-85ad2cdc8713 service placement] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:30:56.469457 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1336a-0408-487a-9d85-85ad2cdc8713 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:30:56.471411 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-64b1336a-0408-487a-9d85-85ad2cdc8713 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:30:56.484246 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1336a-0408-487a-9d85-85ad2cdc8713 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:30:56.485994 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1336a-0408-487a-9d85-85ad2cdc8713 service placement] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:30:56.497440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1422/2864] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:30:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:31:57.221648 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbfd83-4827-44e0-9a36-5e7511524ca6 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:31:57.222563 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbfd83-4827-44e0-9a36-5e7511524ca6 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:31:57.222747 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbfd83-4827-44e0-9a36-5e7511524ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:31:57.242806 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-45fbfd83-4827-44e0-9a36-5e7511524ca6 None None] MFA Rules not processed for user `a5732efd025843a28c9b6948b4b893f5`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:31:57.244259 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-45fbfd83-4827-44e0-9a36-5e7511524ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:31:57.252625 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1443/2865] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:31:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 10:32:00.259027 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1336a-0408-487a-9d85-85ad2cdc8713 service placement] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:32:00.261785 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-64b1336a-0408-487a-9d85-85ad2cdc8713 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:32:00.280873 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1336a-0408-487a-9d85-85ad2cdc8713 service placement] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:32:00.288228 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b885d3-8b28-4844-89b0-7c4ae48be52d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:32:00.289572 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b885d3-8b28-4844-89b0-7c4ae48be52d service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:32:00.290091 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b885d3-8b28-4844-89b0-7c4ae48be52d service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:32:00.290649 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b885d3-8b28-4844-89b0-7c4ae48be52d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:32:00.296188 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e9b885d3-8b28-4844-89b0-7c4ae48be52d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:32:00.403888 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b885d3-8b28-4844-89b0-7c4ae48be52d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:32:00.406654 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b885d3-8b28-4844-89b0-7c4ae48be52d service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:32:00.425371 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1423/2866] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:32:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:32:02.497944 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbfd83-4827-44e0-9a36-5e7511524ca6 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:32:02.522312 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbfd83-4827-44e0-9a36-5e7511524ca6 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:32:02.533035 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47201460-7892-4e5b-9fe4-68ca24dbbb2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:32:02.534619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47201460-7892-4e5b-9fe4-68ca24dbbb2d service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:32:02.535378 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47201460-7892-4e5b-9fe4-68ca24dbbb2d service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:32:02.535861 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47201460-7892-4e5b-9fe4-68ca24dbbb2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:32:02.539047 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-47201460-7892-4e5b-9fe4-68ca24dbbb2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:32:02.564248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47201460-7892-4e5b-9fe4-68ca24dbbb2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:32:02.566682 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47201460-7892-4e5b-9fe4-68ca24dbbb2d service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:32:02.583497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1444/2867] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:32:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:32:10.117342 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b885d3-8b28-4844-89b0-7c4ae48be52d service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:32:10.130554 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b885d3-8b28-4844-89b0-7c4ae48be52d service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:32:10.135131 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dffd38-9fa9-4553-9ec3-a794b1e5635b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:32:10.135812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dffd38-9fa9-4553-9ec3-a794b1e5635b service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:32:10.136247 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dffd38-9fa9-4553-9ec3-a794b1e5635b service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:32:10.136519 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dffd38-9fa9-4553-9ec3-a794b1e5635b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:32:10.138595 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-25dffd38-9fa9-4553-9ec3-a794b1e5635b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:32:10.152226 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dffd38-9fa9-4553-9ec3-a794b1e5635b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:32:10.153782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dffd38-9fa9-4553-9ec3-a794b1e5635b service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:32:10.167596 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1424/2868] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:32:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:32:18.854692 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47201460-7892-4e5b-9fe4-68ca24dbbb2d service nova] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:32:18.857542 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-47201460-7892-4e5b-9fe4-68ca24dbbb2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:32:18.881301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47201460-7892-4e5b-9fe4-68ca24dbbb2d service nova] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:32:18.889723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed265237-aac5-4b94-88c4-7803e1c04ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:32:18.891457 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed265237-aac5-4b94-88c4-7803e1c04ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:32:18.891976 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed265237-aac5-4b94-88c4-7803e1c04ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:32:18.892456 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed265237-aac5-4b94-88c4-7803e1c04ee8 admin admin] PATH_INFO: `/v3/users/ffba3643a2b144739316ea0d5a4c01a9` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:32:18.894997 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed265237-aac5-4b94-88c4-7803e1c04ee8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffba3643a2b144739316ea0d5a4c01a9)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:32:18.899057 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed265237-aac5-4b94-88c4-7803e1c04ee8 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:32:18.980210 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-ed265237-aac5-4b94-88c4-7803e1c04ee8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffba3643a2b144739316ea0d5a4c01a9'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:32:18.988642 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-ed265237-aac5-4b94-88c4-7803e1c04ee8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffba3643a2b144739316ea0d5a4c01a9'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:32:18.989470 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-ed265237-aac5-4b94-88c4-7803e1c04ee8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffba3643a2b144739316ea0d5a4c01a9'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:32:19.002266 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-ed265237-aac5-4b94-88c4-7803e1c04ee8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffba3643a2b144739316ea0d5a4c01a9'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:32:19.003543 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-ed265237-aac5-4b94-88c4-7803e1c04ee8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffba3643a2b144739316ea0d5a4c01a9'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:32:19.025361 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1445/2869] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:32:18 2022] DELETE /identity/v3/users/ffba3643a2b144739316ea0d5a4c01a9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:32:19.284352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dffd38-9fa9-4553-9ec3-a794b1e5635b service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:32:19.308965 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dffd38-9fa9-4553-9ec3-a794b1e5635b service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:32:19.419260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2fc3a-b6e1-4e9b-9de9-fbaab1eefd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:32:19.420305 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2fc3a-b6e1-4e9b-9de9-fbaab1eefd60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:32:19.420766 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2fc3a-b6e1-4e9b-9de9-fbaab1eefd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:32:19.421135 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2fc3a-b6e1-4e9b-9de9-fbaab1eefd60 admin admin] PATH_INFO: `/v3/projects/7b13e936fad64cbcafffb4f896606d4a` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:32:19.422669 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2fc3a-b6e1-4e9b-9de9-fbaab1eefd60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b13e936fad64cbcafffb4f896606d4a)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:32:19.424623 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2fc3a-b6e1-4e9b-9de9-fbaab1eefd60 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:32:19.479458 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e5f2fc3a-b6e1-4e9b-9de9-fbaab1eefd60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b13e936fad64cbcafffb4f896606d4a', 'request_id': 'req-e5f2fc3a-b6e1-4e9b-9de9-fbaab1eefd60'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:32:19.486870 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e5f2fc3a-b6e1-4e9b-9de9-fbaab1eefd60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b13e936fad64cbcafffb4f896606d4a', 'request_id': 'req-e5f2fc3a-b6e1-4e9b-9de9-fbaab1eefd60'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:32:19.493803 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e5f2fc3a-b6e1-4e9b-9de9-fbaab1eefd60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b13e936fad64cbcafffb4f896606d4a', 'request_id': 'req-e5f2fc3a-b6e1-4e9b-9de9-fbaab1eefd60'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:32:19.511483 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e5f2fc3a-b6e1-4e9b-9de9-fbaab1eefd60 admin admin] The token cache is being invalidate because project 7b13e936fad64cbcafffb4f896606d4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:32:19.511894 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e5f2fc3a-b6e1-4e9b-9de9-fbaab1eefd60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:32:19.512786 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-e5f2fc3a-b6e1-4e9b-9de9-fbaab1eefd60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:32:19.514510 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1425/2870] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:32:19 2022] DELETE /identity/v3/projects/7b13e936fad64cbcafffb4f896606d4a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:32:19.518932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed265237-aac5-4b94-88c4-7803e1c04ee8 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:32:19.520701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-ed265237-aac5-4b94-88c4-7803e1c04ee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:32:19.539733 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed265237-aac5-4b94-88c4-7803e1c04ee8 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:32:19.616360 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eae086-60d5-4f94-b9b6-625ef4e8b230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:32:19.617296 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eae086-60d5-4f94-b9b6-625ef4e8b230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:32:19.617570 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eae086-60d5-4f94-b9b6-625ef4e8b230 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:32:19.617839 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eae086-60d5-4f94-b9b6-625ef4e8b230 admin admin] PATH_INFO: `/v3/users/22cce7d2e3a04d60bad56d72ad6e152d` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:32:19.619224 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eae086-60d5-4f94-b9b6-625ef4e8b230 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22cce7d2e3a04d60bad56d72ad6e152d)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:32:19.621237 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eae086-60d5-4f94-b9b6-625ef4e8b230 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:32:19.674335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-78eae086-60d5-4f94-b9b6-625ef4e8b230 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22cce7d2e3a04d60bad56d72ad6e152d'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:32:19.681643 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-78eae086-60d5-4f94-b9b6-625ef4e8b230 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22cce7d2e3a04d60bad56d72ad6e152d'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:32:19.682487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-78eae086-60d5-4f94-b9b6-625ef4e8b230 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22cce7d2e3a04d60bad56d72ad6e152d'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:32:19.691037 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-78eae086-60d5-4f94-b9b6-625ef4e8b230 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22cce7d2e3a04d60bad56d72ad6e152d'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:32:19.691930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-78eae086-60d5-4f94-b9b6-625ef4e8b230 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22cce7d2e3a04d60bad56d72ad6e152d'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:32:19.713007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1446/2871] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:32:19 2022] DELETE /identity/v3/users/22cce7d2e3a04d60bad56d72ad6e152d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:32:19.968118 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2fc3a-b6e1-4e9b-9de9-fbaab1eefd60 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:32:19.970286 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-e5f2fc3a-b6e1-4e9b-9de9-fbaab1eefd60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:32:19.987325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2fc3a-b6e1-4e9b-9de9-fbaab1eefd60 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:32:20.060942 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e85ed-e5ee-48e7-b8ea-c6775295055c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:32:20.061735 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e85ed-e5ee-48e7-b8ea-c6775295055c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:32:20.062065 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e85ed-e5ee-48e7-b8ea-c6775295055c admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:32:20.062267 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e85ed-e5ee-48e7-b8ea-c6775295055c admin admin] PATH_INFO: `/v3/projects/639896afc10345d78ebf3d6e239431f7` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:32:20.063727 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e85ed-e5ee-48e7-b8ea-c6775295055c admin admin] RBAC: Authorizing `identity:delete_project(project_id=639896afc10345d78ebf3d6e239431f7)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:32:20.065761 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e85ed-e5ee-48e7-b8ea-c6775295055c admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:32:20.122860 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-3c3e85ed-e5ee-48e7-b8ea-c6775295055c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '639896afc10345d78ebf3d6e239431f7', 'request_id': 'req-3c3e85ed-e5ee-48e7-b8ea-c6775295055c'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:32:20.131260 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-3c3e85ed-e5ee-48e7-b8ea-c6775295055c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '639896afc10345d78ebf3d6e239431f7', 'request_id': 'req-3c3e85ed-e5ee-48e7-b8ea-c6775295055c'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:32:20.139294 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-3c3e85ed-e5ee-48e7-b8ea-c6775295055c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '639896afc10345d78ebf3d6e239431f7', 'request_id': 'req-3c3e85ed-e5ee-48e7-b8ea-c6775295055c'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:32:20.156515 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-3c3e85ed-e5ee-48e7-b8ea-c6775295055c admin admin] The token cache is being invalidate because project 639896afc10345d78ebf3d6e239431f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:32:20.156839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-3c3e85ed-e5ee-48e7-b8ea-c6775295055c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:32:20.157602 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-3c3e85ed-e5ee-48e7-b8ea-c6775295055c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:32:20.159309 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1426/2872] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:32:19 2022] DELETE /identity/v3/projects/639896afc10345d78ebf3d6e239431f7 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:33:29.802933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49d672-7e4f-47d0-96e8-0154bbe6d5cd None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:33:29.803633 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49d672-7e4f-47d0-96e8-0154bbe6d5cd None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:33:29.803633 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49d672-7e4f-47d0-96e8-0154bbe6d5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:33:29.829453 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-1b49d672-7e4f-47d0-96e8-0154bbe6d5cd None None] MFA Rules not processed for user `a5732efd025843a28c9b6948b4b893f5`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:33:29.830569 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-1b49d672-7e4f-47d0-96e8-0154bbe6d5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:33:29.894115 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1447/2873] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:33:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:33:30.274399 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e85ed-e5ee-48e7-b8ea-c6775295055c admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:33:30.276104 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-3c3e85ed-e5ee-48e7-b8ea-c6775295055c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:33:30.346571 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e85ed-e5ee-48e7-b8ea-c6775295055c admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:33:30.350569 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e56ae-018e-45b0-a70b-37c521a8e47a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:33:30.351246 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e56ae-018e-45b0-a70b-37c521a8e47a service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:33:30.351525 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e56ae-018e-45b0-a70b-37c521a8e47a service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:33:30.351796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e56ae-018e-45b0-a70b-37c521a8e47a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:33:30.353805 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-835e56ae-018e-45b0-a70b-37c521a8e47a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:33:30.364840 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e56ae-018e-45b0-a70b-37c521a8e47a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:33:30.366355 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e56ae-018e-45b0-a70b-37c521a8e47a service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:33:30.376135 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1427/2874] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:33:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:35:05.328416 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d9903-fdce-4f27-afde-9948953a7926 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:35:05.329443 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d9903-fdce-4f27-afde-9948953a7926 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:35:05.329443 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d9903-fdce-4f27-afde-9948953a7926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:35:05.350006 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-bc3d9903-fdce-4f27-afde-9948953a7926 None None] MFA Rules not processed for user `a5732efd025843a28c9b6948b4b893f5`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:35:05.351275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-bc3d9903-fdce-4f27-afde-9948953a7926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:35:05.359028 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1448/2875] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:35:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 10:35:06.108261 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e56ae-018e-45b0-a70b-37c521a8e47a service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:35:06.110399 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-835e56ae-018e-45b0-a70b-37c521a8e47a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:35:06.209395 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e56ae-018e-45b0-a70b-37c521a8e47a service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:35:06.214977 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f3b3e-c8ee-4ecc-9cfc-b7ce3cdb7842 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9e458c2c5f4ed1b3eddd84c2ec1703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:35:06.215627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f3b3e-c8ee-4ecc-9cfc-b7ce3cdb7842 service placement] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:35:06.215932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f3b3e-c8ee-4ecc-9cfc-b7ce3cdb7842 service placement] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:35:06.216178 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f3b3e-c8ee-4ecc-9cfc-b7ce3cdb7842 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:35:06.218654 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-8f9f3b3e-c8ee-4ecc-9cfc-b7ce3cdb7842 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:35:06.235839 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f3b3e-c8ee-4ecc-9cfc-b7ce3cdb7842 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:35:06.237558 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f3b3e-c8ee-4ecc-9cfc-b7ce3cdb7842 service placement] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:35:06.249425 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1428/2876] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:35:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:35:08.371733 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d9903-fdce-4f27-afde-9948953a7926 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:35:08.405475 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d9903-fdce-4f27-afde-9948953a7926 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:35:08.412207 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64cc7f-30b7-4542-80cd-2c4f5b1430d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:35:08.413345 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64cc7f-30b7-4542-80cd-2c4f5b1430d5 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:35:08.413832 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64cc7f-30b7-4542-80cd-2c4f5b1430d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:35:08.414272 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64cc7f-30b7-4542-80cd-2c4f5b1430d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:35:08.417302 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-3a64cc7f-30b7-4542-80cd-2c4f5b1430d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:35:08.459987 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64cc7f-30b7-4542-80cd-2c4f5b1430d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:35:08.462903 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64cc7f-30b7-4542-80cd-2c4f5b1430d5 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:35:08.479477 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1449/2877] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:35:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 10:35:09.637698 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f3b3e-c8ee-4ecc-9cfc-b7ce3cdb7842 service placement] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:35:09.639868 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-8f9f3b3e-c8ee-4ecc-9cfc-b7ce3cdb7842 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:35:09.652994 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f3b3e-c8ee-4ecc-9cfc-b7ce3cdb7842 service placement] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:35:09.657317 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7519a5-0db0-4199-b2aa-0a78be5b1802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:35:09.658129 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7519a5-0db0-4199-b2aa-0a78be5b1802 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:35:09.658504 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7519a5-0db0-4199-b2aa-0a78be5b1802 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:35:09.658838 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7519a5-0db0-4199-b2aa-0a78be5b1802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:35:09.661163 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-db7519a5-0db0-4199-b2aa-0a78be5b1802 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:35:09.782642 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7519a5-0db0-4199-b2aa-0a78be5b1802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:35:09.784304 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7519a5-0db0-4199-b2aa-0a78be5b1802 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:35:09.814453 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1429/2878] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:35:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:35:13.490258 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64cc7f-30b7-4542-80cd-2c4f5b1430d5 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:35:13.500392 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64cc7f-30b7-4542-80cd-2c4f5b1430d5 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:35:13.504976 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428575d6-ece5-4d4b-94fd-b7d74ec7db96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:35:13.505656 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428575d6-ece5-4d4b-94fd-b7d74ec7db96 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:35:13.505937 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428575d6-ece5-4d4b-94fd-b7d74ec7db96 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:35:13.506207 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428575d6-ece5-4d4b-94fd-b7d74ec7db96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:35:13.508355 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-428575d6-ece5-4d4b-94fd-b7d74ec7db96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:35:13.524989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428575d6-ece5-4d4b-94fd-b7d74ec7db96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:35:13.526480 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428575d6-ece5-4d4b-94fd-b7d74ec7db96 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:35:13.540701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1450/2879] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:35:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 10:35:13.766543 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7519a5-0db0-4199-b2aa-0a78be5b1802 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:35:13.769066 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-db7519a5-0db0-4199-b2aa-0a78be5b1802 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:35:13.793247 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7519a5-0db0-4199-b2aa-0a78be5b1802 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:35:13.803637 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e9e8f-527d-4385-985a-ae5f99895027 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:35:13.805111 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e9e8f-527d-4385-985a-ae5f99895027 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:35:13.805610 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e9e8f-527d-4385-985a-ae5f99895027 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:35:13.806171 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e9e8f-527d-4385-985a-ae5f99895027 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:35:13.810014 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-d39e9e8f-527d-4385-985a-ae5f99895027 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:35:13.832316 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e9e8f-527d-4385-985a-ae5f99895027 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:35:13.835295 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e9e8f-527d-4385-985a-ae5f99895027 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:35:13.854090 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1430/2880] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:35:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:35:18.585415 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428575d6-ece5-4d4b-94fd-b7d74ec7db96 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:35:18.587850 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-428575d6-ece5-4d4b-94fd-b7d74ec7db96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:35:18.605252 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428575d6-ece5-4d4b-94fd-b7d74ec7db96 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:35:18.614260 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4652407-ed48-4f86-b64c-9efcd98649c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:35:18.615354 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4652407-ed48-4f86-b64c-9efcd98649c4 service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:35:18.615837 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4652407-ed48-4f86-b64c-9efcd98649c4 service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:35:18.616329 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4652407-ed48-4f86-b64c-9efcd98649c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:35:18.619269 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-e4652407-ed48-4f86-b64c-9efcd98649c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:35:18.642265 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4652407-ed48-4f86-b64c-9efcd98649c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:35:18.645386 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4652407-ed48-4f86-b64c-9efcd98649c4 service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:35:18.665150 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1451/2881] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:35:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:36:00.325868 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e9e8f-527d-4385-985a-ae5f99895027 service nova] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:00.340021 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e9e8f-527d-4385-985a-ae5f99895027 service nova] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:00.347702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff01261-e1e9-422e-a0c1-6f517f91470c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9e458c2c5f4ed1b3eddd84c2ec1703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:00.348474 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff01261-e1e9-422e-a0c1-6f517f91470c service placement] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:00.348844 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff01261-e1e9-422e-a0c1-6f517f91470c service placement] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:00.349172 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff01261-e1e9-422e-a0c1-6f517f91470c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:00.351666 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-eff01261-e1e9-422e-a0c1-6f517f91470c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:36:00.366130 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff01261-e1e9-422e-a0c1-6f517f91470c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:00.367834 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff01261-e1e9-422e-a0c1-6f517f91470c service placement] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:00.380313 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1431/2882] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:36:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:36:23.396599 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4652407-ed48-4f86-b64c-9efcd98649c4 service nova] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:23.408587 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4652407-ed48-4f86-b64c-9efcd98649c4 service nova] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:23.414700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c5a0f-4e68-47ca-b482-64f04ce595a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:23.415369 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c5a0f-4e68-47ca-b482-64f04ce595a1 service nova] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:23.415646 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c5a0f-4e68-47ca-b482-64f04ce595a1 service nova] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:23.415919 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c5a0f-4e68-47ca-b482-64f04ce595a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:23.419176 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-3d5c5a0f-4e68-47ca-b482-64f04ce595a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:36:23.438012 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c5a0f-4e68-47ca-b482-64f04ce595a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:23.440407 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c5a0f-4e68-47ca-b482-64f04ce595a1 service nova] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:23.456590 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1452/2883] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:36:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:36:44.034908 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff01261-e1e9-422e-a0c1-6f517f91470c service placement] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:44.045774 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff01261-e1e9-422e-a0c1-6f517f91470c service placement] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:44.051325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8414a3-d1a0-49a5-b64f-17a5a9c32888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:44.052143 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8414a3-d1a0-49a5-b64f-17a5a9c32888 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:44.052510 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8414a3-d1a0-49a5-b64f-17a5a9c32888 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:44.053022 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8414a3-d1a0-49a5-b64f-17a5a9c32888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:44.055228 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-da8414a3-d1a0-49a5-b64f-17a5a9c32888 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:36:44.154627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8414a3-d1a0-49a5-b64f-17a5a9c32888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:44.156193 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8414a3-d1a0-49a5-b64f-17a5a9c32888 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:44.179363 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1432/2884] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:36:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:36:48.025781 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c5a0f-4e68-47ca-b482-64f04ce595a1 service nova] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:48.027152 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-3d5c5a0f-4e68-47ca-b482-64f04ce595a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:36:48.047139 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c5a0f-4e68-47ca-b482-64f04ce595a1 service nova] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:48.054069 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa294a-b3fb-4d66-81a2-cb9b3456f2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:48.055253 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa294a-b3fb-4d66-81a2-cb9b3456f2a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:48.055533 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa294a-b3fb-4d66-81a2-cb9b3456f2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:48.055804 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa294a-b3fb-4d66-81a2-cb9b3456f2a3 admin admin] PATH_INFO: `/v3/users/a5732efd025843a28c9b6948b4b893f5` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:48.057301 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa294a-b3fb-4d66-81a2-cb9b3456f2a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5732efd025843a28c9b6948b4b893f5)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:48.060488 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa294a-b3fb-4d66-81a2-cb9b3456f2a3 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:48.126999 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-b6aa294a-b3fb-4d66-81a2-cb9b3456f2a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5732efd025843a28c9b6948b4b893f5'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:36:48.134342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-b6aa294a-b3fb-4d66-81a2-cb9b3456f2a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5732efd025843a28c9b6948b4b893f5'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:36:48.135131 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-b6aa294a-b3fb-4d66-81a2-cb9b3456f2a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5732efd025843a28c9b6948b4b893f5'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:36:48.143916 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-b6aa294a-b3fb-4d66-81a2-cb9b3456f2a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5732efd025843a28c9b6948b4b893f5'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:36:48.144627 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-b6aa294a-b3fb-4d66-81a2-cb9b3456f2a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5732efd025843a28c9b6948b4b893f5'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:36:48.166667 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1453/2885] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:36:48 2022] DELETE /identity/v3/users/a5732efd025843a28c9b6948b4b893f5 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 10:36:48.363762 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8414a3-d1a0-49a5-b64f-17a5a9c32888 service neutron] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:48.378472 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8414a3-d1a0-49a5-b64f-17a5a9c32888 service neutron] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:48.440178 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7af922-ecad-445f-9f09-96d4b2ea4528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:48.441075 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7af922-ecad-445f-9f09-96d4b2ea4528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:48.441353 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7af922-ecad-445f-9f09-96d4b2ea4528 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:48.441615 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7af922-ecad-445f-9f09-96d4b2ea4528 admin admin] PATH_INFO: `/v3/projects/6747c07e670e4379874273bc5a88397f` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:48.443025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7af922-ecad-445f-9f09-96d4b2ea4528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6747c07e670e4379874273bc5a88397f)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:48.445025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7af922-ecad-445f-9f09-96d4b2ea4528 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:48.501551 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-3a7af922-ecad-445f-9f09-96d4b2ea4528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6747c07e670e4379874273bc5a88397f', 'request_id': 'req-3a7af922-ecad-445f-9f09-96d4b2ea4528'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:36:48.508025 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-3a7af922-ecad-445f-9f09-96d4b2ea4528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6747c07e670e4379874273bc5a88397f', 'request_id': 'req-3a7af922-ecad-445f-9f09-96d4b2ea4528'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:36:48.513942 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-3a7af922-ecad-445f-9f09-96d4b2ea4528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6747c07e670e4379874273bc5a88397f', 'request_id': 'req-3a7af922-ecad-445f-9f09-96d4b2ea4528'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:36:48.532396 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-3a7af922-ecad-445f-9f09-96d4b2ea4528 admin admin] The token cache is being invalidate because project 6747c07e670e4379874273bc5a88397f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:36:48.533033 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-3a7af922-ecad-445f-9f09-96d4b2ea4528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:36:48.534060 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-3a7af922-ecad-445f-9f09-96d4b2ea4528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:36:48.536762 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1433/2886] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:36:48 2022] DELETE /identity/v3/projects/6747c07e670e4379874273bc5a88397f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 10:36:48.542367 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06e75b-47ea-45ea-97ae-26a465c17324 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:48.542734 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06e75b-47ea-45ea-97ae-26a465c17324 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:48.543072 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06e75b-47ea-45ea-97ae-26a465c17324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:48.607534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-2d06e75b-47ea-45ea-97ae-26a465c17324 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:36:48.608798 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-2d06e75b-47ea-45ea-97ae-26a465c17324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:36:48.671274 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1454/2887] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:36:48.679743 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3e53d-1008-4a2a-97e3-8a2cf6ad93b3 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:48.680000 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3e53d-1008-4a2a-97e3-8a2cf6ad93b3 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:48.680305 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3e53d-1008-4a2a-97e3-8a2cf6ad93b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:48.700215 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-f6f3e53d-1008-4a2a-97e3-8a2cf6ad93b3 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:36:48.701531 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-f6f3e53d-1008-4a2a-97e3-8a2cf6ad93b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:36:48.711038 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1434/2888] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:36:48.715559 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06e75b-47ea-45ea-97ae-26a465c17324 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:48.726352 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06e75b-47ea-45ea-97ae-26a465c17324 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:48.731404 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c133f-95ca-4244-a953-f94170928363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:48.732093 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c133f-95ca-4244-a953-f94170928363 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:48.732369 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c133f-95ca-4244-a953-f94170928363 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:48.732648 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c133f-95ca-4244-a953-f94170928363 admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:48.733431 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c133f-95ca-4244-a953-f94170928363 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 10:36:48.733738 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c133f-95ca-4244-a953-f94170928363 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:48.735347 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c133f-95ca-4244-a953-f94170928363 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:48.748443 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1455/2889] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 10:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 10:36:48.753752 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3e53d-1008-4a2a-97e3-8a2cf6ad93b3 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:48.768953 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3e53d-1008-4a2a-97e3-8a2cf6ad93b3 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:48.775787 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96607e2b-9b77-4352-ab4e-66895b09f709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:48.776925 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96607e2b-9b77-4352-ab4e-66895b09f709 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:48.777381 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96607e2b-9b77-4352-ab4e-66895b09f709 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:48.777859 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96607e2b-9b77-4352-ab4e-66895b09f709 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:48.779146 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96607e2b-9b77-4352-ab4e-66895b09f709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:48.782572 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96607e2b-9b77-4352-ab4e-66895b09f709 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:48.830367 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1435/2890] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:36:48 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:36:48.834708 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c133f-95ca-4244-a953-f94170928363 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:48.845423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c133f-95ca-4244-a953-f94170928363 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:48.909013 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a934c1-106c-4ad4-aeee-1c204eb597e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:48.909780 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a934c1-106c-4ad4-aeee-1c204eb597e4 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:48.910101 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a934c1-106c-4ad4-aeee-1c204eb597e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:48.910374 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a934c1-106c-4ad4-aeee-1c204eb597e4 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:48.911596 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a934c1-106c-4ad4-aeee-1c204eb597e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:48.915020 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a934c1-106c-4ad4-aeee-1c204eb597e4 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:48.943501 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1456/2891] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:36:48 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:36:48.947832 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96607e2b-9b77-4352-ab4e-66895b09f709 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:48.960790 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96607e2b-9b77-4352-ab4e-66895b09f709 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:48.965879 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3de7b3-f264-4418-a641-2a8022f0292d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:48.966553 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3de7b3-f264-4418-a641-2a8022f0292d admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:48.966832 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3de7b3-f264-4418-a641-2a8022f0292d admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:48.967140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3de7b3-f264-4418-a641-2a8022f0292d admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:48.967841 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3de7b3-f264-4418-a641-2a8022f0292d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:48.969523 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3de7b3-f264-4418-a641-2a8022f0292d admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:48.980989 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1436/2892] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 10:36:48.985427 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a934c1-106c-4ad4-aeee-1c204eb597e4 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:48.995897 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a934c1-106c-4ad4-aeee-1c204eb597e4 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:49.000939 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07618c64-4eb9-4712-978a-39fa1e7a6421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:49.001616 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07618c64-4eb9-4712-978a-39fa1e7a6421 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:49.001899 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07618c64-4eb9-4712-978a-39fa1e7a6421 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:49.002196 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07618c64-4eb9-4712-978a-39fa1e7a6421 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:49.002932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07618c64-4eb9-4712-978a-39fa1e7a6421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:49.004574 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07618c64-4eb9-4712-978a-39fa1e7a6421 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:49.015223 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1457/2893] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:36:49.019753 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3de7b3-f264-4418-a641-2a8022f0292d admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:49.030878 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3de7b3-f264-4418-a641-2a8022f0292d admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:49.037173 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7e553-633d-4a4d-a822-c5aa1169b953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:49.038068 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7e553-633d-4a4d-a822-c5aa1169b953 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:49.038338 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7e553-633d-4a4d-a822-c5aa1169b953 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:49.038603 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7e553-633d-4a4d-a822-c5aa1169b953 admin admin] PATH_INFO: `/v3/projects/67eabc70db904ec6b642b32c115cc885/users/41a56eb9859945b49652a7b78b2a2e2d/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:49.054782 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7e553-633d-4a4d-a822-c5aa1169b953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67eabc70db904ec6b642b32c115cc885, user_id=41a56eb9859945b49652a7b78b2a2e2d, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:49.057501 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7e553-633d-4a4d-a822-c5aa1169b953 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:49.071837 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1437/2894] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:36:49 2022] PUT /identity/v3/projects/67eabc70db904ec6b642b32c115cc885/users/41a56eb9859945b49652a7b78b2a2e2d/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:36:49.082955 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0250716-0b29-4aff-b996-d374f0d05b43 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:49.083568 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0250716-0b29-4aff-b996-d374f0d05b43 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:49.083568 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0250716-0b29-4aff-b996-d374f0d05b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:49.116008 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-b0250716-0b29-4aff-b996-d374f0d05b43 None None] MFA Rules not processed for user `41a56eb9859945b49652a7b78b2a2e2d`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:36:49.117285 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-b0250716-0b29-4aff-b996-d374f0d05b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:36:49.194680 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1458/2895] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:36:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:36:49.206486 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7e553-633d-4a4d-a822-c5aa1169b953 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:49.208440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-6ee7e553-633d-4a4d-a822-c5aa1169b953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:36:49.289878 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7e553-633d-4a4d-a822-c5aa1169b953 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:49.297041 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88016829-c680-4365-b854-c9537b498325 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6362303cb918486c8525225914332ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:49.297825 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88016829-c680-4365-b854-c9537b498325 service nova] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:49.298190 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88016829-c680-4365-b854-c9537b498325 service nova] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:49.298573 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88016829-c680-4365-b854-c9537b498325 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:49.308136 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88016829-c680-4365-b854-c9537b498325 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:49.310108 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88016829-c680-4365-b854-c9537b498325 service nova] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:49.320101 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1438/2896] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:36:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:36:52.097149 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0250716-0b29-4aff-b996-d374f0d05b43 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:52.187895 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0250716-0b29-4aff-b996-d374f0d05b43 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:52.194612 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66901b55-b9a6-4858-b6f2-72e00dbfa5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:52.195514 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66901b55-b9a6-4858-b6f2-72e00dbfa5e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:52.195791 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66901b55-b9a6-4858-b6f2-72e00dbfa5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:52.196153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66901b55-b9a6-4858-b6f2-72e00dbfa5e9 admin admin] PATH_INFO: `/v3/users/41a56eb9859945b49652a7b78b2a2e2d` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:52.197595 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66901b55-b9a6-4858-b6f2-72e00dbfa5e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41a56eb9859945b49652a7b78b2a2e2d)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:52.199580 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66901b55-b9a6-4858-b6f2-72e00dbfa5e9 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:52.261516 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-66901b55-b9a6-4858-b6f2-72e00dbfa5e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41a56eb9859945b49652a7b78b2a2e2d'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:36:52.270011 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-66901b55-b9a6-4858-b6f2-72e00dbfa5e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41a56eb9859945b49652a7b78b2a2e2d'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:36:52.270909 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-66901b55-b9a6-4858-b6f2-72e00dbfa5e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41a56eb9859945b49652a7b78b2a2e2d'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:36:52.281159 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-66901b55-b9a6-4858-b6f2-72e00dbfa5e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41a56eb9859945b49652a7b78b2a2e2d'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:36:52.282072 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-66901b55-b9a6-4858-b6f2-72e00dbfa5e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41a56eb9859945b49652a7b78b2a2e2d'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:36:52.302764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1459/2897] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:36:52 2022] DELETE /identity/v3/users/41a56eb9859945b49652a7b78b2a2e2d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:36:52.307887 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e00e3-6022-4423-9155-a334f5557bfc None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:52.308718 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e00e3-6022-4423-9155-a334f5557bfc None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:52.308718 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e00e3-6022-4423-9155-a334f5557bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:52.328796 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-493e00e3-6022-4423-9155-a334f5557bfc None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:36:52.331016 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-493e00e3-6022-4423-9155-a334f5557bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:36:52.436996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1439/2898] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:36:52.448472 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441659b-41aa-4c25-968c-bb3c6accc821 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:52.448935 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441659b-41aa-4c25-968c-bb3c6accc821 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:52.449303 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441659b-41aa-4c25-968c-bb3c6accc821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:52.485502 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-6441659b-41aa-4c25-968c-bb3c6accc821 None None] MFA Rules not processed for user `0f0d3f4264a04093894d6bc5c5c45c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:36:52.486826 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-6441659b-41aa-4c25-968c-bb3c6accc821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:36:52.551579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1460/2899] 10.222.0.26 () {58 vars in 1061 bytes} [Mon May 9 10:36:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:36:52.559919 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e00e3-6022-4423-9155-a334f5557bfc None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:52.572021 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e00e3-6022-4423-9155-a334f5557bfc None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:52.577034 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a47f6-1810-422c-83bc-8ad64699e7a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:52.577816 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a47f6-1810-422c-83bc-8ad64699e7a2 service neutron] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:52.578204 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a47f6-1810-422c-83bc-8ad64699e7a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:52.578576 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a47f6-1810-422c-83bc-8ad64699e7a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:52.589777 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a47f6-1810-422c-83bc-8ad64699e7a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:52.591273 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a47f6-1810-422c-83bc-8ad64699e7a2 service neutron] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:52.602550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1440/2900] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:36:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:36:52.932081 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441659b-41aa-4c25-968c-bb3c6accc821 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:52.933758 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-6441659b-41aa-4c25-968c-bb3c6accc821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:36:52.952447 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441659b-41aa-4c25-968c-bb3c6accc821 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:52.959497 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f676f-048d-4127-ae5e-bf3649848ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:52.960556 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f676f-048d-4127-ae5e-bf3649848ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:52.961030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f676f-048d-4127-ae5e-bf3649848ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:52.961374 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f676f-048d-4127-ae5e-bf3649848ccc admin admin] PATH_INFO: `/v3/projects/67eabc70db904ec6b642b32c115cc885` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:52.963183 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f676f-048d-4127-ae5e-bf3649848ccc admin admin] RBAC: Authorizing `identity:delete_project(project_id=67eabc70db904ec6b642b32c115cc885)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:52.967317 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f676f-048d-4127-ae5e-bf3649848ccc admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:53.038045 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9b2f676f-048d-4127-ae5e-bf3649848ccc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67eabc70db904ec6b642b32c115cc885', 'request_id': 'req-9b2f676f-048d-4127-ae5e-bf3649848ccc'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:36:53.045468 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9b2f676f-048d-4127-ae5e-bf3649848ccc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67eabc70db904ec6b642b32c115cc885', 'request_id': 'req-9b2f676f-048d-4127-ae5e-bf3649848ccc'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:36:53.051966 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9b2f676f-048d-4127-ae5e-bf3649848ccc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67eabc70db904ec6b642b32c115cc885', 'request_id': 'req-9b2f676f-048d-4127-ae5e-bf3649848ccc'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:36:53.067335 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9b2f676f-048d-4127-ae5e-bf3649848ccc admin admin] The token cache is being invalidate because project 67eabc70db904ec6b642b32c115cc885 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86598) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:36:53.067677 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9b2f676f-048d-4127-ae5e-bf3649848ccc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:36:53.068418 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-9b2f676f-048d-4127-ae5e-bf3649848ccc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:36:53.070287 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1461/2901] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:36:52 2022] DELETE /identity/v3/projects/67eabc70db904ec6b642b32c115cc885 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:36:53.075744 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f733a7-2341-4926-9248-5092669f54f5 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:53.075981 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f733a7-2341-4926-9248-5092669f54f5 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:53.076275 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f733a7-2341-4926-9248-5092669f54f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:53.097282 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-69f733a7-2341-4926-9248-5092669f54f5 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:36:53.098409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-69f733a7-2341-4926-9248-5092669f54f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:36:53.167279 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1441/2902] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 10:36:53.228584 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17060e73-1704-41d3-be1e-8a0bf102160c None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:53.229337 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17060e73-1704-41d3-be1e-8a0bf102160c None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:53.229337 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17060e73-1704-41d3-be1e-8a0bf102160c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:53.250714 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-17060e73-1704-41d3-be1e-8a0bf102160c None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:36:53.252345 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-17060e73-1704-41d3-be1e-8a0bf102160c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:36:53.261973 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1462/2903] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:36:53.266691 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f733a7-2341-4926-9248-5092669f54f5 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:53.277565 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f733a7-2341-4926-9248-5092669f54f5 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:53.283128 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc0545-5c09-428e-a03c-61bc05852c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:53.283920 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc0545-5c09-428e-a03c-61bc05852c18 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:53.284299 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc0545-5c09-428e-a03c-61bc05852c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:53.284687 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc0545-5c09-428e-a03c-61bc05852c18 admin admin] PATH_INFO: `/v3/domains` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:53.285614 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc0545-5c09-428e-a03c-61bc05852c18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86597) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 10:36:53.286005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc0545-5c09-428e-a03c-61bc05852c18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:53.287740 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc0545-5c09-428e-a03c-61bc05852c18 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:53.301853 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1442/2904] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 10:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 10:36:53.306700 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17060e73-1704-41d3-be1e-8a0bf102160c None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:53.317653 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17060e73-1704-41d3-be1e-8a0bf102160c None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:53.322718 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d22b0-539b-484b-ae47-fbe81d5bf54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:53.323393 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d22b0-539b-484b-ae47-fbe81d5bf54e admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:53.323668 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d22b0-539b-484b-ae47-fbe81d5bf54e admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:53.323908 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d22b0-539b-484b-ae47-fbe81d5bf54e admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:53.324678 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d22b0-539b-484b-ae47-fbe81d5bf54e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:53.326649 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d22b0-539b-484b-ae47-fbe81d5bf54e admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:53.357616 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1463/2905] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:36:53 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:36:53.362752 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc0545-5c09-428e-a03c-61bc05852c18 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:53.380550 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc0545-5c09-428e-a03c-61bc05852c18 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:53.451353 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf412ce-c894-41cb-9b1b-50b800ec7f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:53.452028 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf412ce-c894-41cb-9b1b-50b800ec7f16 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:53.452305 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf412ce-c894-41cb-9b1b-50b800ec7f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:53.452581 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf412ce-c894-41cb-9b1b-50b800ec7f16 admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:53.453306 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf412ce-c894-41cb-9b1b-50b800ec7f16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:53.455382 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf412ce-c894-41cb-9b1b-50b800ec7f16 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:53.477511 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1443/2906] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:36:53 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:36:53.483591 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d22b0-539b-484b-ae47-fbe81d5bf54e admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:53.495743 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d22b0-539b-484b-ae47-fbe81d5bf54e admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:53.501296 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ba226-8b55-4ecb-86e8-0c103436ae1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:53.502843 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ba226-8b55-4ecb-86e8-0c103436ae1f admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:53.503162 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ba226-8b55-4ecb-86e8-0c103436ae1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:53.503472 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ba226-8b55-4ecb-86e8-0c103436ae1f admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:53.504273 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ba226-8b55-4ecb-86e8-0c103436ae1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:53.506030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ba226-8b55-4ecb-86e8-0c103436ae1f admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:53.518010 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1464/2907] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 10:36:53.522296 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf412ce-c894-41cb-9b1b-50b800ec7f16 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:53.533055 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf412ce-c894-41cb-9b1b-50b800ec7f16 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:53.538628 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d106780-6474-4c9f-a45b-404fcb19a9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:53.539648 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d106780-6474-4c9f-a45b-404fcb19a9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:53.540071 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d106780-6474-4c9f-a45b-404fcb19a9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:53.540721 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d106780-6474-4c9f-a45b-404fcb19a9b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:53.541865 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d106780-6474-4c9f-a45b-404fcb19a9b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:53.544356 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d106780-6474-4c9f-a45b-404fcb19a9b8 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:53.557794 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1444/2908] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:36:53.563207 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ba226-8b55-4ecb-86e8-0c103436ae1f admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:53.574938 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ba226-8b55-4ecb-86e8-0c103436ae1f admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:53.581151 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9b32a-e4ba-4eb6-9a19-55e93d1aa4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:53.582236 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9b32a-e4ba-4eb6-9a19-55e93d1aa4b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:53.582868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9b32a-e4ba-4eb6-9a19-55e93d1aa4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:53.583303 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9b32a-e4ba-4eb6-9a19-55e93d1aa4b9 admin admin] PATH_INFO: `/v3/projects/38f50d3b9b3e40c8b54740cb81354736/users/e11be8542a9d479a982c0f7475e43ddc/roles/fec647282f5a42ab8490975708efad14` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:53.601746 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9b32a-e4ba-4eb6-9a19-55e93d1aa4b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38f50d3b9b3e40c8b54740cb81354736, user_id=e11be8542a9d479a982c0f7475e43ddc, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:53.604473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9b32a-e4ba-4eb6-9a19-55e93d1aa4b9 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:53.619397 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1465/2909] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:36:53 2022] PUT /identity/v3/projects/38f50d3b9b3e40c8b54740cb81354736/users/e11be8542a9d479a982c0f7475e43ddc/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:36:53.630326 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ebf00-aa16-4af7-8d5c-d7bf997b717a None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:53.630645 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ebf00-aa16-4af7-8d5c-d7bf997b717a None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:53.630864 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ebf00-aa16-4af7-8d5c-d7bf997b717a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:53.663098 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-409ebf00-aa16-4af7-8d5c-d7bf997b717a None None] MFA Rules not processed for user `e11be8542a9d479a982c0f7475e43ddc`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:36:53.664380 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-409ebf00-aa16-4af7-8d5c-d7bf997b717a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:36:53.737442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1445/2910] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:36:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 10:36:53.742400 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9b32a-e4ba-4eb6-9a19-55e93d1aa4b9 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:53.812479 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9b32a-e4ba-4eb6-9a19-55e93d1aa4b9 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:53.817730 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37cb26d-0c58-4071-a85a-1ee365b4c606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:53.818421 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37cb26d-0c58-4071-a85a-1ee365b4c606 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:53.818684 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37cb26d-0c58-4071-a85a-1ee365b4c606 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:53.818950 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37cb26d-0c58-4071-a85a-1ee365b4c606 admin admin] PATH_INFO: `/v3/projects` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:53.819670 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37cb26d-0c58-4071-a85a-1ee365b4c606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:53.821655 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37cb26d-0c58-4071-a85a-1ee365b4c606 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:53.851173 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1466/2911] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:36:53 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:36:53.855974 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ebf00-aa16-4af7-8d5c-d7bf997b717a None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:53.868641 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ebf00-aa16-4af7-8d5c-d7bf997b717a None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:53.951329 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2045c-ff7e-4aad-b18a-35b1f2139fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:53.951972 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2045c-ff7e-4aad-b18a-35b1f2139fde admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:53.952249 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2045c-ff7e-4aad-b18a-35b1f2139fde admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:53.952523 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2045c-ff7e-4aad-b18a-35b1f2139fde admin admin] PATH_INFO: `/v3/users` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:53.953315 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2045c-ff7e-4aad-b18a-35b1f2139fde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:53.955439 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2045c-ff7e-4aad-b18a-35b1f2139fde admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:53.978888 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1446/2912] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:36:53 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:36:53.983649 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37cb26d-0c58-4071-a85a-1ee365b4c606 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:53.995703 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37cb26d-0c58-4071-a85a-1ee365b4c606 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:54.002025 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb96a3-d743-450f-855d-04da3a2f4660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:54.002811 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb96a3-d743-450f-855d-04da3a2f4660 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:54.003126 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb96a3-d743-450f-855d-04da3a2f4660 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:54.003432 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb96a3-d743-450f-855d-04da3a2f4660 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:54.004222 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb96a3-d743-450f-855d-04da3a2f4660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:54.005994 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb96a3-d743-450f-855d-04da3a2f4660 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:54.018693 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1467/2913] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:36:54.023628 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2045c-ff7e-4aad-b18a-35b1f2139fde admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:54.034692 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2045c-ff7e-4aad-b18a-35b1f2139fde admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:54.039724 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d5c77-83d2-4987-9d51-244100368185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:54.040638 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d5c77-83d2-4987-9d51-244100368185 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:54.040956 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d5c77-83d2-4987-9d51-244100368185 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:54.041230 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d5c77-83d2-4987-9d51-244100368185 admin admin] PATH_INFO: `/v3/projects/c20222d2f4004761aa36269e8a8600a4/users/ab313056a0af4a08be62e15861f95b26/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:54.059333 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d5c77-83d2-4987-9d51-244100368185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c20222d2f4004761aa36269e8a8600a4, user_id=ab313056a0af4a08be62e15861f95b26, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:54.062390 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d5c77-83d2-4987-9d51-244100368185 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:54.076571 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1447/2914] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:36:54 2022] PUT /identity/v3/projects/c20222d2f4004761aa36269e8a8600a4/users/ab313056a0af4a08be62e15861f95b26/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:36:54.081266 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb96a3-d743-450f-855d-04da3a2f4660 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:54.099434 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb96a3-d743-450f-855d-04da3a2f4660 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:54.200248 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ec740-5620-4b00-a21c-079db7eabe88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:54.201488 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ec740-5620-4b00-a21c-079db7eabe88 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:54.201899 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ec740-5620-4b00-a21c-079db7eabe88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:54.202689 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ec740-5620-4b00-a21c-079db7eabe88 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:54.204059 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ec740-5620-4b00-a21c-079db7eabe88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:54.207575 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ec740-5620-4b00-a21c-079db7eabe88 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:54.227863 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1468/2915] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:36:54.232478 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d5c77-83d2-4987-9d51-244100368185 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:54.243789 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d5c77-83d2-4987-9d51-244100368185 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:54.249314 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c1fde-d0ef-481f-b195-a14e4fa01f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:54.250243 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c1fde-d0ef-481f-b195-a14e4fa01f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:54.250517 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c1fde-d0ef-481f-b195-a14e4fa01f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:54.250785 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c1fde-d0ef-481f-b195-a14e4fa01f91 admin admin] PATH_INFO: `/v3/projects/c20222d2f4004761aa36269e8a8600a4/users/ab313056a0af4a08be62e15861f95b26/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:54.253257 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c1fde-d0ef-481f-b195-a14e4fa01f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c20222d2f4004761aa36269e8a8600a4, user_id=ab313056a0af4a08be62e15861f95b26, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:54.256324 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c1fde-d0ef-481f-b195-a14e4fa01f91 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:54.268348 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1448/2916] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:36:54 2022] PUT /identity/v3/projects/c20222d2f4004761aa36269e8a8600a4/users/ab313056a0af4a08be62e15861f95b26/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:36:54.279608 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd891c31-11a1-4b25-b568-2062ae666b3f None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:54.280438 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd891c31-11a1-4b25-b568-2062ae666b3f None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:54.280597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd891c31-11a1-4b25-b568-2062ae666b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:54.312361 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-cd891c31-11a1-4b25-b568-2062ae666b3f None None] MFA Rules not processed for user `ab313056a0af4a08be62e15861f95b26`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:36:54.313656 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-cd891c31-11a1-4b25-b568-2062ae666b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:36:54.400843 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1469/2917] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:36:54 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:36:54.413501 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c1fde-d0ef-481f-b195-a14e4fa01f91 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:54.415494 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-537c1fde-d0ef-481f-b195-a14e4fa01f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:36:54.518759 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c1fde-d0ef-481f-b195-a14e4fa01f91 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:54.527712 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9963cb-df94-47fe-84bd-94e6ad268de9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:54.528501 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9963cb-df94-47fe-84bd-94e6ad268de9 service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:54.528869 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9963cb-df94-47fe-84bd-94e6ad268de9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:54.529210 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9963cb-df94-47fe-84bd-94e6ad268de9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:54.543979 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9963cb-df94-47fe-84bd-94e6ad268de9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:54.546320 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9963cb-df94-47fe-84bd-94e6ad268de9 service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:54.557618 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1449/2918] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:36:54 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:36:54.651842 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd891c31-11a1-4b25-b568-2062ae666b3f None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:36:54.663060 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd891c31-11a1-4b25-b568-2062ae666b3f None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:36:54.667822 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db563a03-a0ab-4ff0-bbe2-3c09e9d35f63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:36:54.668493 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db563a03-a0ab-4ff0-bbe2-3c09e9d35f63 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:36:54.668813 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db563a03-a0ab-4ff0-bbe2-3c09e9d35f63 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:36:54.669095 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db563a03-a0ab-4ff0-bbe2-3c09e9d35f63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:36:54.740181 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db563a03-a0ab-4ff0-bbe2-3c09e9d35f63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:36:54.741733 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db563a03-a0ab-4ff0-bbe2-3c09e9d35f63 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:36:54.752097 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1470/2919] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:36:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:37:38.855996 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9963cb-df94-47fe-84bd-94e6ad268de9 service cinder] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:38.936136 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9963cb-df94-47fe-84bd-94e6ad268de9 service cinder] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:38.942503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b7acc-24ca-474c-be7c-afa1e0367d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:38.943492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b7acc-24ca-474c-be7c-afa1e0367d4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:38.944024 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b7acc-24ca-474c-be7c-afa1e0367d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:38.944197 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b7acc-24ca-474c-be7c-afa1e0367d4f admin admin] PATH_INFO: `/v3/users/e11be8542a9d479a982c0f7475e43ddc` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:38.945593 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b7acc-24ca-474c-be7c-afa1e0367d4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e11be8542a9d479a982c0f7475e43ddc)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:38.947980 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b7acc-24ca-474c-be7c-afa1e0367d4f admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:39.007549 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-d35b7acc-24ca-474c-be7c-afa1e0367d4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e11be8542a9d479a982c0f7475e43ddc'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:39.016943 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-d35b7acc-24ca-474c-be7c-afa1e0367d4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e11be8542a9d479a982c0f7475e43ddc'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:39.026659 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-d35b7acc-24ca-474c-be7c-afa1e0367d4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e11be8542a9d479a982c0f7475e43ddc'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:39.027437 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-d35b7acc-24ca-474c-be7c-afa1e0367d4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e11be8542a9d479a982c0f7475e43ddc'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:39.028127 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-d35b7acc-24ca-474c-be7c-afa1e0367d4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e11be8542a9d479a982c0f7475e43ddc'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:39.051288 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1450/2920] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:37:38 2022] DELETE /identity/v3/users/e11be8542a9d479a982c0f7475e43ddc => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:37:39.056168 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ae00f-a1ab-4798-a666-1fdd844ce8ec None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:39.056808 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ae00f-a1ab-4798-a666-1fdd844ce8ec None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:39.056917 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ae00f-a1ab-4798-a666-1fdd844ce8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:39.076151 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-275ae00f-a1ab-4798-a666-1fdd844ce8ec None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:37:39.077681 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-275ae00f-a1ab-4798-a666-1fdd844ce8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:37:39.150357 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1471/2921] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:37:39.162983 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7b009-8cb8-4f2b-9fb7-8e7207e49e02 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:39.163308 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7b009-8cb8-4f2b-9fb7-8e7207e49e02 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:39.163645 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7b009-8cb8-4f2b-9fb7-8e7207e49e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:39.184601 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-baf7b009-8cb8-4f2b-9fb7-8e7207e49e02 None None] MFA Rules not processed for user `0f0d3f4264a04093894d6bc5c5c45c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:37:39.186421 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-baf7b009-8cb8-4f2b-9fb7-8e7207e49e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:37:39.249982 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1451/2922] 10.222.0.26 () {58 vars in 1061 bytes} [Mon May 9 10:37:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:37:39.256975 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ae00f-a1ab-4798-a666-1fdd844ce8ec None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:39.267868 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ae00f-a1ab-4798-a666-1fdd844ce8ec None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:39.272294 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73830e3-b5a2-4b12-9e3e-106d945ae3f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:39.273007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73830e3-b5a2-4b12-9e3e-106d945ae3f3 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:39.273293 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73830e3-b5a2-4b12-9e3e-106d945ae3f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:39.273567 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73830e3-b5a2-4b12-9e3e-106d945ae3f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:39.285957 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73830e3-b5a2-4b12-9e3e-106d945ae3f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:39.288790 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73830e3-b5a2-4b12-9e3e-106d945ae3f3 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:39.307063 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1472/2923] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:37:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:37:39.681915 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7b009-8cb8-4f2b-9fb7-8e7207e49e02 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:39.683472 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-baf7b009-8cb8-4f2b-9fb7-8e7207e49e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:37:39.698070 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7b009-8cb8-4f2b-9fb7-8e7207e49e02 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:39.702889 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695b9d0-6303-4e44-950f-8ebcfa559a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:39.703781 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695b9d0-6303-4e44-950f-8ebcfa559a20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:39.704056 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695b9d0-6303-4e44-950f-8ebcfa559a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:39.704326 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695b9d0-6303-4e44-950f-8ebcfa559a20 admin admin] PATH_INFO: `/v3/projects/38f50d3b9b3e40c8b54740cb81354736` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:39.705956 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0695b9d0-6303-4e44-950f-8ebcfa559a20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38f50d3b9b3e40c8b54740cb81354736)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:39.708073 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0695b9d0-6303-4e44-950f-8ebcfa559a20 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:39.767180 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0695b9d0-6303-4e44-950f-8ebcfa559a20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38f50d3b9b3e40c8b54740cb81354736', 'request_id': 'req-0695b9d0-6303-4e44-950f-8ebcfa559a20'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:39.774500 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0695b9d0-6303-4e44-950f-8ebcfa559a20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38f50d3b9b3e40c8b54740cb81354736', 'request_id': 'req-0695b9d0-6303-4e44-950f-8ebcfa559a20'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:39.780901 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0695b9d0-6303-4e44-950f-8ebcfa559a20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38f50d3b9b3e40c8b54740cb81354736', 'request_id': 'req-0695b9d0-6303-4e44-950f-8ebcfa559a20'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:39.795437 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0695b9d0-6303-4e44-950f-8ebcfa559a20 admin admin] The token cache is being invalidate because project 38f50d3b9b3e40c8b54740cb81354736 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:37:39.795712 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0695b9d0-6303-4e44-950f-8ebcfa559a20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:39.796384 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-0695b9d0-6303-4e44-950f-8ebcfa559a20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:39.798112 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1452/2924] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:37:39 2022] DELETE /identity/v3/projects/38f50d3b9b3e40c8b54740cb81354736 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 10:37:39.802874 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73830e3-b5a2-4b12-9e3e-106d945ae3f3 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:39.813843 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73830e3-b5a2-4b12-9e3e-106d945ae3f3 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:39.885040 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69903fdf-747c-4789-97ca-176c8036301e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:39.885940 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69903fdf-747c-4789-97ca-176c8036301e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:39.886216 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69903fdf-747c-4789-97ca-176c8036301e admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:39.886487 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69903fdf-747c-4789-97ca-176c8036301e admin admin] PATH_INFO: `/v3/users/ab313056a0af4a08be62e15861f95b26` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:39.887819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69903fdf-747c-4789-97ca-176c8036301e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab313056a0af4a08be62e15861f95b26)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:39.889885 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69903fdf-747c-4789-97ca-176c8036301e admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:39.943933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-69903fdf-747c-4789-97ca-176c8036301e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab313056a0af4a08be62e15861f95b26'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:39.951988 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-69903fdf-747c-4789-97ca-176c8036301e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab313056a0af4a08be62e15861f95b26'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:39.952792 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-69903fdf-747c-4789-97ca-176c8036301e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab313056a0af4a08be62e15861f95b26'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:39.961752 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-69903fdf-747c-4789-97ca-176c8036301e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab313056a0af4a08be62e15861f95b26'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:39.962545 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-69903fdf-747c-4789-97ca-176c8036301e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab313056a0af4a08be62e15861f95b26'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:39.981781 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1473/2925] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:37:39 2022] DELETE /identity/v3/users/ab313056a0af4a08be62e15861f95b26 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:37:40.292486 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695b9d0-6303-4e44-950f-8ebcfa559a20 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:40.294457 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-0695b9d0-6303-4e44-950f-8ebcfa559a20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:37:40.313126 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695b9d0-6303-4e44-950f-8ebcfa559a20 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:40.381629 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38854ffa-2104-4796-928e-d8eee714636d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:40.382507 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38854ffa-2104-4796-928e-d8eee714636d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:40.382783 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38854ffa-2104-4796-928e-d8eee714636d admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:40.383058 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38854ffa-2104-4796-928e-d8eee714636d admin admin] PATH_INFO: `/v3/projects/c20222d2f4004761aa36269e8a8600a4` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:40.384326 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38854ffa-2104-4796-928e-d8eee714636d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c20222d2f4004761aa36269e8a8600a4)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:40.386352 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38854ffa-2104-4796-928e-d8eee714636d admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:40.436925 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-38854ffa-2104-4796-928e-d8eee714636d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c20222d2f4004761aa36269e8a8600a4', 'request_id': 'req-38854ffa-2104-4796-928e-d8eee714636d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:40.443704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-38854ffa-2104-4796-928e-d8eee714636d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c20222d2f4004761aa36269e8a8600a4', 'request_id': 'req-38854ffa-2104-4796-928e-d8eee714636d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:40.449529 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-38854ffa-2104-4796-928e-d8eee714636d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c20222d2f4004761aa36269e8a8600a4', 'request_id': 'req-38854ffa-2104-4796-928e-d8eee714636d'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:40.464198 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-38854ffa-2104-4796-928e-d8eee714636d admin admin] The token cache is being invalidate because project c20222d2f4004761aa36269e8a8600a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:37:40.464526 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-38854ffa-2104-4796-928e-d8eee714636d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:40.465249 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-38854ffa-2104-4796-928e-d8eee714636d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:40.466862 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1453/2926] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:37:40 2022] DELETE /identity/v3/projects/c20222d2f4004761aa36269e8a8600a4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:37:40.472061 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a26bf-741f-40d0-bce3-a902e664ba61 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:40.472943 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a26bf-741f-40d0-bce3-a902e664ba61 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:40.472943 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a26bf-741f-40d0-bce3-a902e664ba61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:40.493625 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-2b8a26bf-741f-40d0-bce3-a902e664ba61 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:37:40.494975 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-2b8a26bf-741f-40d0-bce3-a902e664ba61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:37:40.562752 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1474/2927] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:37:40.571295 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee577d6-704f-47a9-a45c-a4de73b7a0e3 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:40.571561 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee577d6-704f-47a9-a45c-a4de73b7a0e3 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:40.571864 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee577d6-704f-47a9-a45c-a4de73b7a0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:40.592294 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-dee577d6-704f-47a9-a45c-a4de73b7a0e3 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:37:40.593587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-dee577d6-704f-47a9-a45c-a4de73b7a0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:37:40.602546 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1454/2928] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:37:40.607249 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a26bf-741f-40d0-bce3-a902e664ba61 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:40.617767 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a26bf-741f-40d0-bce3-a902e664ba61 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:40.623344 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749b0f1-b881-49ac-86a9-339f6e192ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:40.624096 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749b0f1-b881-49ac-86a9-339f6e192ddc admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:40.624418 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749b0f1-b881-49ac-86a9-339f6e192ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:40.624796 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749b0f1-b881-49ac-86a9-339f6e192ddc admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:40.625587 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1749b0f1-b881-49ac-86a9-339f6e192ddc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 10:37:40.625956 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1749b0f1-b881-49ac-86a9-339f6e192ddc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:40.627783 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1749b0f1-b881-49ac-86a9-339f6e192ddc admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:40.641290 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1475/2929] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 10:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 10:37:40.645722 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee577d6-704f-47a9-a45c-a4de73b7a0e3 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:40.655591 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee577d6-704f-47a9-a45c-a4de73b7a0e3 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:40.661447 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8366e-df60-47ef-af5b-145151e1cc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:40.662337 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8366e-df60-47ef-af5b-145151e1cc75 admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:40.662708 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8366e-df60-47ef-af5b-145151e1cc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:40.663069 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8366e-df60-47ef-af5b-145151e1cc75 admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:40.663895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f8366e-df60-47ef-af5b-145151e1cc75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:40.666063 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f8366e-df60-47ef-af5b-145151e1cc75 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:40.696559 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1455/2930] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:37:40 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 10:37:40.701052 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749b0f1-b881-49ac-86a9-339f6e192ddc admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:40.712213 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749b0f1-b881-49ac-86a9-339f6e192ddc admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:40.777622 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce7edb-d7ac-4a86-a6ee-462c540156da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:40.778382 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce7edb-d7ac-4a86-a6ee-462c540156da admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:40.778766 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce7edb-d7ac-4a86-a6ee-462c540156da admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:40.779094 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce7edb-d7ac-4a86-a6ee-462c540156da admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:40.779855 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce7edb-d7ac-4a86-a6ee-462c540156da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:40.782030 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce7edb-d7ac-4a86-a6ee-462c540156da admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:40.804920 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1476/2931] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:37:40 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:37:40.809533 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8366e-df60-47ef-af5b-145151e1cc75 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:40.821807 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8366e-df60-47ef-af5b-145151e1cc75 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:40.828017 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a10033-440e-434d-897c-15577846fb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:40.828841 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a10033-440e-434d-897c-15577846fb15 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:40.829026 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a10033-440e-434d-897c-15577846fb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:40.829324 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a10033-440e-434d-897c-15577846fb15 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:40.830068 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a10033-440e-434d-897c-15577846fb15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:40.831840 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a10033-440e-434d-897c-15577846fb15 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:40.844233 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1456/2932] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:37:40.849007 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce7edb-d7ac-4a86-a6ee-462c540156da admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:40.860204 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce7edb-d7ac-4a86-a6ee-462c540156da admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:40.865619 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9accdd83-3ba0-45f9-9e25-57edb6cba524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:40.866425 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9accdd83-3ba0-45f9-9e25-57edb6cba524 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:40.866798 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9accdd83-3ba0-45f9-9e25-57edb6cba524 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:40.867153 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9accdd83-3ba0-45f9-9e25-57edb6cba524 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:40.867982 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9accdd83-3ba0-45f9-9e25-57edb6cba524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:40.869932 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9accdd83-3ba0-45f9-9e25-57edb6cba524 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:40.882206 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1477/2933] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:37:40.886290 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a10033-440e-434d-897c-15577846fb15 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:40.897409 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a10033-440e-434d-897c-15577846fb15 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:40.902641 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423461e-a419-463f-9150-dff7498536be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:40.903615 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423461e-a419-463f-9150-dff7498536be admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:40.903885 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423461e-a419-463f-9150-dff7498536be admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:40.904154 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423461e-a419-463f-9150-dff7498536be admin admin] PATH_INFO: `/v3/projects/dfaf7ac1e37b424981fe7940f6890b49/users/b56e6074ffa54194a62b8fbba4f664aa/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:40.921132 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423461e-a419-463f-9150-dff7498536be admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfaf7ac1e37b424981fe7940f6890b49, user_id=b56e6074ffa54194a62b8fbba4f664aa, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:40.923960 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423461e-a419-463f-9150-dff7498536be admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:40.940398 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1457/2934] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:37:40 2022] PUT /identity/v3/projects/dfaf7ac1e37b424981fe7940f6890b49/users/b56e6074ffa54194a62b8fbba4f664aa/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:37:40.952146 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecec162-8bd7-45aa-bafb-45e626d524d8 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:40.952495 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecec162-8bd7-45aa-bafb-45e626d524d8 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:40.952759 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecec162-8bd7-45aa-bafb-45e626d524d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:40.982635 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-fecec162-8bd7-45aa-bafb-45e626d524d8 None None] MFA Rules not processed for user `b56e6074ffa54194a62b8fbba4f664aa`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:37:40.983889 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-fecec162-8bd7-45aa-bafb-45e626d524d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:37:41.055279 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1478/2935] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:37:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:37:41.060034 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423461e-a419-463f-9150-dff7498536be admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:41.128944 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423461e-a419-463f-9150-dff7498536be admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:41.134100 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b8525-f4f2-4b3f-af13-8a5953aca32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:41.134776 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b8525-f4f2-4b3f-af13-8a5953aca32f admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:41.135055 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b8525-f4f2-4b3f-af13-8a5953aca32f admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:41.135326 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b8525-f4f2-4b3f-af13-8a5953aca32f admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:41.136029 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b8525-f4f2-4b3f-af13-8a5953aca32f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:41.138005 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b8525-f4f2-4b3f-af13-8a5953aca32f admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:41.168746 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1458/2936] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:37:41 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:37:41.173238 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecec162-8bd7-45aa-bafb-45e626d524d8 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:41.184855 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecec162-8bd7-45aa-bafb-45e626d524d8 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:41.255510 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe099f1f-56da-4506-9e18-400d0665319b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:41.256224 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe099f1f-56da-4506-9e18-400d0665319b admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:41.256506 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe099f1f-56da-4506-9e18-400d0665319b admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:41.256819 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe099f1f-56da-4506-9e18-400d0665319b admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:41.257554 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe099f1f-56da-4506-9e18-400d0665319b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:41.259565 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe099f1f-56da-4506-9e18-400d0665319b admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:41.282423 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1479/2937] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:37:41 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:37:41.287271 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b8525-f4f2-4b3f-af13-8a5953aca32f admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:41.299218 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b8525-f4f2-4b3f-af13-8a5953aca32f admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:41.305492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6541b03-001f-419f-af9e-6cecb46f64d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:41.306180 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6541b03-001f-419f-af9e-6cecb46f64d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:41.306492 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6541b03-001f-419f-af9e-6cecb46f64d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:41.306731 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6541b03-001f-419f-af9e-6cecb46f64d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:41.307493 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6541b03-001f-419f-af9e-6cecb46f64d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:41.309145 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6541b03-001f-419f-af9e-6cecb46f64d7 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:41.321821 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1459/2938] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:37:41.326633 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe099f1f-56da-4506-9e18-400d0665319b admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:41.337246 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe099f1f-56da-4506-9e18-400d0665319b admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:41.342453 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633248ed-cd48-47ca-8231-044f710af9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:41.343381 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633248ed-cd48-47ca-8231-044f710af9e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:41.343656 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633248ed-cd48-47ca-8231-044f710af9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:41.343927 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633248ed-cd48-47ca-8231-044f710af9e2 admin admin] PATH_INFO: `/v3/projects/87b512e0946b4a79b2059a3fec70efdf/users/b52a9b2b9cb54e67a05676ef01180e59/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:41.362285 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633248ed-cd48-47ca-8231-044f710af9e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87b512e0946b4a79b2059a3fec70efdf, user_id=b52a9b2b9cb54e67a05676ef01180e59, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:41.365577 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633248ed-cd48-47ca-8231-044f710af9e2 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:41.382977 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1480/2939] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:37:41 2022] PUT /identity/v3/projects/87b512e0946b4a79b2059a3fec70efdf/users/b52a9b2b9cb54e67a05676ef01180e59/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 10:37:41.387278 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6541b03-001f-419f-af9e-6cecb46f64d7 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:41.398400 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6541b03-001f-419f-af9e-6cecb46f64d7 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:41.466325 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0dca7-3dd3-4eae-8f92-9e7e63442931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:41.467015 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0dca7-3dd3-4eae-8f92-9e7e63442931 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:41.467301 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0dca7-3dd3-4eae-8f92-9e7e63442931 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:41.467572 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0dca7-3dd3-4eae-8f92-9e7e63442931 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:41.468320 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0dca7-3dd3-4eae-8f92-9e7e63442931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:41.470012 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0dca7-3dd3-4eae-8f92-9e7e63442931 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:41.481113 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1460/2940] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:37:41.485761 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633248ed-cd48-47ca-8231-044f710af9e2 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:41.497579 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633248ed-cd48-47ca-8231-044f710af9e2 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:41.502789 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea978dbb-8f14-4ce4-a942-787315fcd951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:41.503704 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea978dbb-8f14-4ce4-a942-787315fcd951 admin admin] REQUEST_METHOD: `PUT` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:41.503981 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea978dbb-8f14-4ce4-a942-787315fcd951 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:41.504251 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea978dbb-8f14-4ce4-a942-787315fcd951 admin admin] PATH_INFO: `/v3/projects/87b512e0946b4a79b2059a3fec70efdf/users/b52a9b2b9cb54e67a05676ef01180e59/roles/70ac549382b14b2cbb8b35e5dfc648d4` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:41.506458 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea978dbb-8f14-4ce4-a942-787315fcd951 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87b512e0946b4a79b2059a3fec70efdf, user_id=b52a9b2b9cb54e67a05676ef01180e59, role_id=70ac549382b14b2cbb8b35e5dfc648d4)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:41.509796 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea978dbb-8f14-4ce4-a942-787315fcd951 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:41.525036 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1481/2941] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:37:41 2022] PUT /identity/v3/projects/87b512e0946b4a79b2059a3fec70efdf/users/b52a9b2b9cb54e67a05676ef01180e59/roles/70ac549382b14b2cbb8b35e5dfc648d4 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:37:41.538682 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ebd7b-08ff-47bd-bd80-25f0e8a5176b None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:41.538967 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ebd7b-08ff-47bd-bd80-25f0e8a5176b None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:41.539253 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ebd7b-08ff-47bd-bd80-25f0e8a5176b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:41.570503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-857ebd7b-08ff-47bd-bd80-25f0e8a5176b None None] MFA Rules not processed for user `b52a9b2b9cb54e67a05676ef01180e59`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:37:41.571838 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-857ebd7b-08ff-47bd-bd80-25f0e8a5176b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:37:41.649105 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1461/2942] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:37:41 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:37:41.661560 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea978dbb-8f14-4ce4-a942-787315fcd951 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:41.663472 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-ea978dbb-8f14-4ce4-a942-787315fcd951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:37:41.728256 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea978dbb-8f14-4ce4-a942-787315fcd951 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:41.732464 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe897db-d407-47b0-83c2-d38fc23d9367 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:41.733168 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe897db-d407-47b0-83c2-d38fc23d9367 service cinder] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:41.733447 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe897db-d407-47b0-83c2-d38fc23d9367 service cinder] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:41.733721 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe897db-d407-47b0-83c2-d38fc23d9367 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:41.743990 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe897db-d407-47b0-83c2-d38fc23d9367 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:41.745503 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe897db-d407-47b0-83c2-d38fc23d9367 service cinder] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:41.756838 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1482/2943] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:37:41 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:37:41.837812 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ebd7b-08ff-47bd-bd80-25f0e8a5176b None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:41.849751 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ebd7b-08ff-47bd-bd80-25f0e8a5176b None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:41.854510 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18b6a3-ae76-4a60-94fa-07b24f63c72b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:41.855178 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18b6a3-ae76-4a60-94fa-07b24f63c72b service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:41.855450 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18b6a3-ae76-4a60-94fa-07b24f63c72b service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:41.855716 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18b6a3-ae76-4a60-94fa-07b24f63c72b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:41.922254 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b18b6a3-ae76-4a60-94fa-07b24f63c72b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:41.923690 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b18b6a3-ae76-4a60-94fa-07b24f63c72b service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:41.934259 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1462/2944] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:37:41 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:37:44.848702 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe897db-d407-47b0-83c2-d38fc23d9367 service cinder] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:44.918168 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe897db-d407-47b0-83c2-d38fc23d9367 service cinder] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:44.922934 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e9555-67ca-446a-b55e-f33a7aad4adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:44.923808 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e9555-67ca-446a-b55e-f33a7aad4adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:44.924082 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e9555-67ca-446a-b55e-f33a7aad4adb admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:44.924372 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e9555-67ca-446a-b55e-f33a7aad4adb admin admin] PATH_INFO: `/v3/users/b56e6074ffa54194a62b8fbba4f664aa` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:44.925817 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e9555-67ca-446a-b55e-f33a7aad4adb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b56e6074ffa54194a62b8fbba4f664aa)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:44.927805 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e9555-67ca-446a-b55e-f33a7aad4adb admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:44.983772 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-488e9555-67ca-446a-b55e-f33a7aad4adb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b56e6074ffa54194a62b8fbba4f664aa'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:44.992290 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-488e9555-67ca-446a-b55e-f33a7aad4adb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b56e6074ffa54194a62b8fbba4f664aa'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:44.993165 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-488e9555-67ca-446a-b55e-f33a7aad4adb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b56e6074ffa54194a62b8fbba4f664aa'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:45.003502 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-488e9555-67ca-446a-b55e-f33a7aad4adb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b56e6074ffa54194a62b8fbba4f664aa'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:45.004216 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-488e9555-67ca-446a-b55e-f33a7aad4adb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b56e6074ffa54194a62b8fbba4f664aa'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:45.025965 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1483/2945] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:37:44 2022] DELETE /identity/v3/users/b56e6074ffa54194a62b8fbba4f664aa => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:37:45.030702 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458572b-5e96-47a8-b1c9-7d96aa9b24ff None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:45.031405 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458572b-5e96-47a8-b1c9-7d96aa9b24ff None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:45.031405 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458572b-5e96-47a8-b1c9-7d96aa9b24ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:45.051140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-b458572b-5e96-47a8-b1c9-7d96aa9b24ff None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:37:45.052425 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-b458572b-5e96-47a8-b1c9-7d96aa9b24ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:37:45.123411 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1463/2946] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 10:37:45.134887 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e9555-67ca-446a-b55e-f33a7aad4adb admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:45.136444 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-488e9555-67ca-446a-b55e-f33a7aad4adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:37:45.207271 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e9555-67ca-446a-b55e-f33a7aad4adb admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:45.211764 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b83b26-5ce2-4765-aba4-c5ef6759743e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:45.212637 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b83b26-5ce2-4765-aba4-c5ef6759743e service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:45.213002 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b83b26-5ce2-4765-aba4-c5ef6759743e service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:45.213345 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b83b26-5ce2-4765-aba4-c5ef6759743e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:45.225545 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b83b26-5ce2-4765-aba4-c5ef6759743e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:45.227353 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b83b26-5ce2-4765-aba4-c5ef6759743e service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:45.239405 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1484/2947] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:37:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:37:45.577116 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458572b-5e96-47a8-b1c9-7d96aa9b24ff None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:45.578964 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-b458572b-5e96-47a8-b1c9-7d96aa9b24ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:37:45.593650 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458572b-5e96-47a8-b1c9-7d96aa9b24ff None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:45.602229 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057abca5-f118-4d58-ad72-0bb3fe5c3e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:45.603847 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057abca5-f118-4d58-ad72-0bb3fe5c3e9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:45.604406 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057abca5-f118-4d58-ad72-0bb3fe5c3e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:45.604900 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057abca5-f118-4d58-ad72-0bb3fe5c3e9b admin admin] PATH_INFO: `/v3/projects/dfaf7ac1e37b424981fe7940f6890b49` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:45.608215 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057abca5-f118-4d58-ad72-0bb3fe5c3e9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfaf7ac1e37b424981fe7940f6890b49)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:45.612235 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057abca5-f118-4d58-ad72-0bb3fe5c3e9b admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:45.669895 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-057abca5-f118-4d58-ad72-0bb3fe5c3e9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfaf7ac1e37b424981fe7940f6890b49', 'request_id': 'req-057abca5-f118-4d58-ad72-0bb3fe5c3e9b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:45.678074 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-057abca5-f118-4d58-ad72-0bb3fe5c3e9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfaf7ac1e37b424981fe7940f6890b49', 'request_id': 'req-057abca5-f118-4d58-ad72-0bb3fe5c3e9b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:45.684883 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-057abca5-f118-4d58-ad72-0bb3fe5c3e9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfaf7ac1e37b424981fe7940f6890b49', 'request_id': 'req-057abca5-f118-4d58-ad72-0bb3fe5c3e9b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:45.700894 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-057abca5-f118-4d58-ad72-0bb3fe5c3e9b admin admin] The token cache is being invalidate because project dfaf7ac1e37b424981fe7940f6890b49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:37:45.701204 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-057abca5-f118-4d58-ad72-0bb3fe5c3e9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:45.701975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-057abca5-f118-4d58-ad72-0bb3fe5c3e9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:45.703723 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1464/2948] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:37:45 2022] DELETE /identity/v3/projects/dfaf7ac1e37b424981fe7940f6890b49 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:37:45.709830 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b83b26-5ce2-4765-aba4-c5ef6759743e service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:45.720271 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b83b26-5ce2-4765-aba4-c5ef6759743e service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:45.803361 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a18fe6-0daf-4dbe-872a-d4179ada7ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:45.804257 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a18fe6-0daf-4dbe-872a-d4179ada7ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:45.804534 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a18fe6-0daf-4dbe-872a-d4179ada7ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:45.804847 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a18fe6-0daf-4dbe-872a-d4179ada7ee9 admin admin] PATH_INFO: `/v3/users/b52a9b2b9cb54e67a05676ef01180e59` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:45.806116 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a18fe6-0daf-4dbe-872a-d4179ada7ee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b52a9b2b9cb54e67a05676ef01180e59)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:45.808118 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a18fe6-0daf-4dbe-872a-d4179ada7ee9 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:45.860172 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e7a18fe6-0daf-4dbe-872a-d4179ada7ee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b52a9b2b9cb54e67a05676ef01180e59'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:45.868287 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e7a18fe6-0daf-4dbe-872a-d4179ada7ee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b52a9b2b9cb54e67a05676ef01180e59'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:45.869049 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e7a18fe6-0daf-4dbe-872a-d4179ada7ee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b52a9b2b9cb54e67a05676ef01180e59'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:45.878767 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e7a18fe6-0daf-4dbe-872a-d4179ada7ee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b52a9b2b9cb54e67a05676ef01180e59'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:45.879418 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-e7a18fe6-0daf-4dbe-872a-d4179ada7ee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b52a9b2b9cb54e67a05676ef01180e59'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:45.900723 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1485/2949] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:37:45 2022] DELETE /identity/v3/users/b52a9b2b9cb54e67a05676ef01180e59 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:37:46.280258 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057abca5-f118-4d58-ad72-0bb3fe5c3e9b admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:46.283194 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-057abca5-f118-4d58-ad72-0bb3fe5c3e9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:37:46.305206 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057abca5-f118-4d58-ad72-0bb3fe5c3e9b admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:46.379144 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15444ce1-bcce-41e0-826b-e2317b0f893b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:46.380584 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15444ce1-bcce-41e0-826b-e2317b0f893b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:46.380584 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15444ce1-bcce-41e0-826b-e2317b0f893b admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:46.380932 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15444ce1-bcce-41e0-826b-e2317b0f893b admin admin] PATH_INFO: `/v3/projects/87b512e0946b4a79b2059a3fec70efdf` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:46.382481 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15444ce1-bcce-41e0-826b-e2317b0f893b admin admin] RBAC: Authorizing `identity:delete_project(project_id=87b512e0946b4a79b2059a3fec70efdf)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:46.384956 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15444ce1-bcce-41e0-826b-e2317b0f893b admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:46.440875 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-15444ce1-bcce-41e0-826b-e2317b0f893b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87b512e0946b4a79b2059a3fec70efdf', 'request_id': 'req-15444ce1-bcce-41e0-826b-e2317b0f893b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:46.448140 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-15444ce1-bcce-41e0-826b-e2317b0f893b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87b512e0946b4a79b2059a3fec70efdf', 'request_id': 'req-15444ce1-bcce-41e0-826b-e2317b0f893b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:46.455221 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-15444ce1-bcce-41e0-826b-e2317b0f893b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87b512e0946b4a79b2059a3fec70efdf', 'request_id': 'req-15444ce1-bcce-41e0-826b-e2317b0f893b'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:46.470247 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-15444ce1-bcce-41e0-826b-e2317b0f893b admin admin] The token cache is being invalidate because project 87b512e0946b4a79b2059a3fec70efdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:37:46.470537 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-15444ce1-bcce-41e0-826b-e2317b0f893b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:46.471388 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-15444ce1-bcce-41e0-826b-e2317b0f893b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:46.473389 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1465/2950] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:37:46 2022] DELETE /identity/v3/projects/87b512e0946b4a79b2059a3fec70efdf => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:37:46.480747 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50646c-230a-4287-aff5-beb5a8576356 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:46.481032 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50646c-230a-4287-aff5-beb5a8576356 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:46.481308 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50646c-230a-4287-aff5-beb5a8576356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:46.501290 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-7a50646c-230a-4287-aff5-beb5a8576356 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:37:46.502606 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-7a50646c-230a-4287-aff5-beb5a8576356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:37:46.574072 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1486/2951] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 10:37:46.582544 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d2822-27d8-47c1-ab9b-4dc0f10c1566 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:46.582801 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d2822-27d8-47c1-ab9b-4dc0f10c1566 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:46.583184 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d2822-27d8-47c1-ab9b-4dc0f10c1566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:46.603539 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-d30d2822-27d8-47c1-ab9b-4dc0f10c1566 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:37:46.605378 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-d30d2822-27d8-47c1-ab9b-4dc0f10c1566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:37:46.614669 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1466/2952] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 10:37:46.619601 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50646c-230a-4287-aff5-beb5a8576356 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:46.630082 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50646c-230a-4287-aff5-beb5a8576356 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:46.635550 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1786b-0fea-4cfe-bc0e-fca3d7c4da69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:46.636292 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1786b-0fea-4cfe-bc0e-fca3d7c4da69 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:46.636606 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1786b-0fea-4cfe-bc0e-fca3d7c4da69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:46.636960 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1786b-0fea-4cfe-bc0e-fca3d7c4da69 admin admin] PATH_INFO: `/v3/domains` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:46.637796 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1786b-0fea-4cfe-bc0e-fca3d7c4da69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86598) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 10:37:46.638132 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1786b-0fea-4cfe-bc0e-fca3d7c4da69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:46.639848 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1786b-0fea-4cfe-bc0e-fca3d7c4da69 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:46.653022 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1487/2953] 10.222.0.26 () {58 vars in 1152 bytes} [Mon May 9 10:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 10:37:46.659470 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d2822-27d8-47c1-ab9b-4dc0f10c1566 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:46.669475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d2822-27d8-47c1-ab9b-4dc0f10c1566 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:46.675274 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e8dd1-75ff-4114-9000-d8d111c756df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:46.676076 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e8dd1-75ff-4114-9000-d8d111c756df admin admin] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:46.676513 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e8dd1-75ff-4114-9000-d8d111c756df admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:46.676878 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e8dd1-75ff-4114-9000-d8d111c756df admin admin] PATH_INFO: `/v3/projects` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:46.677704 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e8dd1-75ff-4114-9000-d8d111c756df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:46.679898 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e8dd1-75ff-4114-9000-d8d111c756df admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:46.713252 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1467/2954] 10.222.0.26 () {60 vars in 1152 bytes} [Mon May 9 10:37:46 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 10:37:46.717934 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1786b-0fea-4cfe-bc0e-fca3d7c4da69 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:46.731981 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1786b-0fea-4cfe-bc0e-fca3d7c4da69 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:46.831662 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c2935-fe05-49fe-a199-6a9d15829928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:46.832811 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c2935-fe05-49fe-a199-6a9d15829928 admin admin] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:46.833237 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c2935-fe05-49fe-a199-6a9d15829928 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:46.833717 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c2935-fe05-49fe-a199-6a9d15829928 admin admin] PATH_INFO: `/v3/users` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:46.834978 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c2935-fe05-49fe-a199-6a9d15829928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:46.838776 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c2935-fe05-49fe-a199-6a9d15829928 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:46.873074 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1488/2955] 10.222.0.26 () {60 vars in 1143 bytes} [Mon May 9 10:37:46 2022] POST /identity/v3/users => generated 288 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 10:37:46.879487 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e8dd1-75ff-4114-9000-d8d111c756df admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:46.891334 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e8dd1-75ff-4114-9000-d8d111c756df admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:46.898045 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d8916-ebb5-469f-8468-881cd03eabb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:46.898852 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d8916-ebb5-469f-8468-881cd03eabb8 admin admin] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:46.899223 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d8916-ebb5-469f-8468-881cd03eabb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:46.899582 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d8916-ebb5-469f-8468-881cd03eabb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:46.900451 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d8916-ebb5-469f-8468-881cd03eabb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:46.902299 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d8916-ebb5-469f-8468-881cd03eabb8 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:46.919240 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1468/2956] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:37:46.923535 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c2935-fe05-49fe-a199-6a9d15829928 admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:46.935047 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c2935-fe05-49fe-a199-6a9d15829928 admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:46.941158 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0155be0-f656-449d-918d-7c4f705fdff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:46.941930 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0155be0-f656-449d-918d-7c4f705fdff2 admin admin] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:46.942275 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0155be0-f656-449d-918d-7c4f705fdff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:46.942584 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0155be0-f656-449d-918d-7c4f705fdff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:46.943384 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0155be0-f656-449d-918d-7c4f705fdff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:46.945284 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0155be0-f656-449d-918d-7c4f705fdff2 admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:46.958086 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1489/2957] 10.222.0.26 () {58 vars in 1121 bytes} [Mon May 9 10:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 10:37:46.962999 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d8916-ebb5-469f-8468-881cd03eabb8 admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:46.975243 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d8916-ebb5-469f-8468-881cd03eabb8 admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:46.982551 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66d296-090c-4b71-b101-c44b15098c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:46.984034 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66d296-090c-4b71-b101-c44b15098c6b admin admin] REQUEST_METHOD: `PUT` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:46.984490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66d296-090c-4b71-b101-c44b15098c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:46.984963 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66d296-090c-4b71-b101-c44b15098c6b admin admin] PATH_INFO: `/v3/projects/772bbc8ddcc64befaf24daac9cb9085a/users/519e556e9530496091c5327d40feb880/roles/fec647282f5a42ab8490975708efad14` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:47.004288 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66d296-090c-4b71-b101-c44b15098c6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=772bbc8ddcc64befaf24daac9cb9085a, user_id=519e556e9530496091c5327d40feb880, role_id=fec647282f5a42ab8490975708efad14)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:47.007006 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66d296-090c-4b71-b101-c44b15098c6b admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:47.023365 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1469/2958] 10.222.0.26 () {60 vars in 1482 bytes} [Mon May 9 10:37:46 2022] PUT /identity/v3/projects/772bbc8ddcc64befaf24daac9cb9085a/users/519e556e9530496091c5327d40feb880/roles/fec647282f5a42ab8490975708efad14 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:37:47.035629 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62effd-38e4-43b0-b8c8-71da70d36a07 None None] REQUEST_METHOD: `POST` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:47.035933 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62effd-38e4-43b0-b8c8-71da70d36a07 None None] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:47.036235 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62effd-38e4-43b0-b8c8-71da70d36a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:47.068296 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.auth.core [None req-5e62effd-38e4-43b0-b8c8-71da70d36a07 None None] MFA Rules not processed for user `519e556e9530496091c5327d40feb880`. Rule list: `[]` (Enabled: `True`). {{(pid=86598) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:37:47.069849 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-5e62effd-38e4-43b0-b8c8-71da70d36a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:37:47.157541 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1490/2959] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:37:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 10:37:47.169626 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66d296-090c-4b71-b101-c44b15098c6b admin admin] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:47.171475 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-be66d296-090c-4b71-b101-c44b15098c6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:37:47.243440 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66d296-090c-4b71-b101-c44b15098c6b admin admin] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:47.250237 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab1bbd-51f2-4e6f-8288-a757f0b15f3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a44d689993bd4d30858a770012ee3e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:47.251376 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab1bbd-51f2-4e6f-8288-a757f0b15f3c service cinder] REQUEST_METHOD: `GET` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:47.251877 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab1bbd-51f2-4e6f-8288-a757f0b15f3c service cinder] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:47.252356 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab1bbd-51f2-4e6f-8288-a757f0b15f3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:47.268944 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab1bbd-51f2-4e6f-8288-a757f0b15f3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:47.271866 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab1bbd-51f2-4e6f-8288-a757f0b15f3c service cinder] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:47.289700 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1470/2960] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:37:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 10:37:50.471263 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62effd-38e4-43b0-b8c8-71da70d36a07 None None] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:50.557342 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62effd-38e4-43b0-b8c8-71da70d36a07 None None] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:50.566801 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4e7bd-de66-4c2d-80c8-1ea39e90772c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:50.567713 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4e7bd-de66-4c2d-80c8-1ea39e90772c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:50.567989 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4e7bd-de66-4c2d-80c8-1ea39e90772c admin admin] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:50.568262 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4e7bd-de66-4c2d-80c8-1ea39e90772c admin admin] PATH_INFO: `/v3/users/519e556e9530496091c5327d40feb880` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:50.569696 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4e7bd-de66-4c2d-80c8-1ea39e90772c admin admin] RBAC: Authorizing `identity:delete_user(user_id=519e556e9530496091c5327d40feb880)` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:50.571785 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4e7bd-de66-4c2d-80c8-1ea39e90772c admin admin] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:50.628548 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-18d4e7bd-de66-4c2d-80c8-1ea39e90772c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '519e556e9530496091c5327d40feb880'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:50.636782 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-18d4e7bd-de66-4c2d-80c8-1ea39e90772c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '519e556e9530496091c5327d40feb880'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:50.637549 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-18d4e7bd-de66-4c2d-80c8-1ea39e90772c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '519e556e9530496091c5327d40feb880'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:50.647192 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-18d4e7bd-de66-4c2d-80c8-1ea39e90772c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '519e556e9530496091c5327d40feb880'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:50.647883 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.notifications [None req-18d4e7bd-de66-4c2d-80c8-1ea39e90772c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '519e556e9530496091c5327d40feb880'} {{(pid=86598) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:50.668839 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1491/2961] 10.222.0.26 () {58 vars in 1223 bytes} [Mon May 9 10:37:50 2022] DELETE /identity/v3/users/519e556e9530496091c5327d40feb880 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:37:50.674076 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c5d1d-0c24-4787-a5c6-9c0dfc177cc3 None None] REQUEST_METHOD: `POST` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:50.674975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c5d1d-0c24-4787-a5c6-9c0dfc177cc3 None None] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:50.674975 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c5d1d-0c24-4787-a5c6-9c0dfc177cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:50.694679 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.auth.core [None req-482c5d1d-0c24-4787-a5c6-9c0dfc177cc3 None None] MFA Rules not processed for user `68d9ab86ee454d7d9d1e4d3a48048d52`. Rule list: `[]` (Enabled: `True`). {{(pid=86597) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 10:37:50.695847 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-482c5d1d-0c24-4787-a5c6-9c0dfc177cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:37:50.782587 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1471/2962] 10.222.0.26 () {58 vars in 957 bytes} [Mon May 9 10:37:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 10:37:50.795632 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4e7bd-de66-4c2d-80c8-1ea39e90772c admin admin] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:50.797276 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-18d4e7bd-de66-4c2d-80c8-1ea39e90772c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:37:50.864311 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4e7bd-de66-4c2d-80c8-1ea39e90772c admin admin] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:50.868215 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e0f6c-f155-4669-821f-e45855d94a11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f0d3f4264a04093894d6bc5c5c45c7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:50.868917 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e0f6c-f155-4669-821f-e45855d94a11 service neutron] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:50.869190 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e0f6c-f155-4669-821f-e45855d94a11 service neutron] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:50.869473 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e0f6c-f155-4669-821f-e45855d94a11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:50.880284 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e0f6c-f155-4669-821f-e45855d94a11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:50.881856 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e0f6c-f155-4669-821f-e45855d94a11 service neutron] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:50.895194 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1492/2963] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:37:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 10:37:51.219448 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c5d1d-0c24-4787-a5c6-9c0dfc177cc3 None None] Authenticating user token {{(pid=86597) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:37:51.221442 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.fernet_utils [None req-482c5d1d-0c24-4787-a5c6-9c0dfc177cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86597) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:37:51.242253 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c5d1d-0c24-4787-a5c6-9c0dfc177cc3 None None] Validating token access rules against request {{(pid=86597) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:37:51.249134 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef71c8f-2ba1-4b7e-8a78-6d06794bc702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d9ab86ee454d7d9d1e4d3a48048d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03d8c548fb514d70b477284ceb7b3252', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86597) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:37:51.250070 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef71c8f-2ba1-4b7e-8a78-6d06794bc702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:37:51.250503 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef71c8f-2ba1-4b7e-8a78-6d06794bc702 admin admin] SCRIPT_NAME: `/identity` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:37:51.250934 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef71c8f-2ba1-4b7e-8a78-6d06794bc702 admin admin] PATH_INFO: `/v3/projects/772bbc8ddcc64befaf24daac9cb9085a` {{(pid=86597) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:37:51.253453 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef71c8f-2ba1-4b7e-8a78-6d06794bc702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=772bbc8ddcc64befaf24daac9cb9085a)` {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:37:51.257627 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef71c8f-2ba1-4b7e-8a78-6d06794bc702 admin admin] RBAC: Authorization granted {{(pid=86597) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:37:51.324696 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-1ef71c8f-2ba1-4b7e-8a78-6d06794bc702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '772bbc8ddcc64befaf24daac9cb9085a', 'request_id': 'req-1ef71c8f-2ba1-4b7e-8a78-6d06794bc702'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:51.332411 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-1ef71c8f-2ba1-4b7e-8a78-6d06794bc702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '772bbc8ddcc64befaf24daac9cb9085a', 'request_id': 'req-1ef71c8f-2ba1-4b7e-8a78-6d06794bc702'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:51.340826 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-1ef71c8f-2ba1-4b7e-8a78-6d06794bc702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '772bbc8ddcc64befaf24daac9cb9085a', 'request_id': 'req-1ef71c8f-2ba1-4b7e-8a78-6d06794bc702'} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:51.357046 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-1ef71c8f-2ba1-4b7e-8a78-6d06794bc702 admin admin] The token cache is being invalidate because project 772bbc8ddcc64befaf24daac9cb9085a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86597) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 10:37:51.357490 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-1ef71c8f-2ba1-4b7e-8a78-6d06794bc702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:51.358354 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: DEBUG keystone.notifications [None req-1ef71c8f-2ba1-4b7e-8a78-6d06794bc702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86597) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 10:37:51.360217 dv-cinder-841083-1-iscsi devstack@keystone.service[86597]: [pid: 86597|app: 0|req: 1472/2964] 10.222.0.26 () {58 vars in 1232 bytes} [Mon May 9 10:37:51 2022] DELETE /identity/v3/projects/772bbc8ddcc64befaf24daac9cb9085a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 10:41:04.303877 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e0f6c-f155-4669-821f-e45855d94a11 service neutron] Authenticating user token {{(pid=86598) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 10:41:04.305654 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-4b3e0f6c-f155-4669-821f-e45855d94a11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:41:04.468701 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e0f6c-f155-4669-821f-e45855d94a11 service neutron] Validating token access rules against request {{(pid=86598) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 10:41:04.536241 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad477c0-c8d8-4f37-bd9b-9c049755c03c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9e458c2c5f4ed1b3eddd84c2ec1703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cd39464055141b8b43fde20d0885c99', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86598) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 10:41:04.536946 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad477c0-c8d8-4f37-bd9b-9c049755c03c service placement] REQUEST_METHOD: `GET` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 10:41:04.537229 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad477c0-c8d8-4f37-bd9b-9c049755c03c service placement] SCRIPT_NAME: `/identity` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 10:41:04.537489 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad477c0-c8d8-4f37-bd9b-9c049755c03c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86598) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 10:41:04.539459 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.fernet_utils [None req-fad477c0-c8d8-4f37-bd9b-9c049755c03c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86598) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 10:41:04.552170 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad477c0-c8d8-4f37-bd9b-9c049755c03c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 10:41:04.553788 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad477c0-c8d8-4f37-bd9b-9c049755c03c service placement] RBAC: Authorization granted {{(pid=86598) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 10:41:04.568597 dv-cinder-841083-1-iscsi devstack@keystone.service[86598]: [pid: 86598|app: 0|req: 1493/2965] 10.222.0.26 () {60 vars in 1365 bytes} [Mon May 9 10:41:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)